JOIN_CACHE's were initialized in check_join_cache_usage()
from make_join_readinfo(). After that make_join_readinfo() was looking
whether it's possible to use keyread. Later, after make_join_readinfo(),
optimizer decided whether to use filesort. And even later, at the
execution time, from join_read_first(), keyread was actually enabled.
The problem is, that if a query uses a vcol, base columns that it
depends on are automatically added to the read_set - because they're
needed to calculate the vcol. But if we're doing keyread, vcol is taken
from the index, not calculated, and base columns do not need to be
in the read set (even should not be - as they aren't getting values).
The bug was that JOIN_CACHE used read_set with base columns,
they were not read because of keyread, so it was caching garbage.
So read_set is only known after the keyread was decided. And after the
filesort was decided, as filesort doesn't use keyread. But
check_join_cache_usage() needs to be done in make_join_readinfo(),
as the code below depends on these checks,
Fix: keep JOIN_CACHE checks where they were, but move initialization
down to the very end of JOIN::optimize_inner. If keyread was enabled,
update the read_set to include only columns that are part of the index.
Copy the keyread logic from join_read_first() to happen at optimize time.
move TABLE::key_read into handler. Because in index merge and DS-MRR
there can be many handlers per table, and some of them use
key read while others don't. "keyread" is really per handler,
not per TABLE property.
Oracle introduced a Memcached plugin interface to the InnoDB
storage engine in MySQL 5.6. That interface is essentially a
fork of Memcached development snapshot 1.6.0-beta1 of an old
development branch 'engine-pu'.
To my knowledge, there have not been any updates to the Memcached code
between MySQL 5.6 and 5.7; only bug fixes and extensions related to
the Oracle modifications.
The Memcached plugin is not part of the MariaDB Server. Therefore it
does not make sense to include the InnoDB interfaces for the Memcached
plugin, or to have any related configuration parameters:
innodb_api_bk_commit_interval
innodb_api_disable_rowlock
innodb_api_enable_binlog
innodb_api_enable_mdl
innodb_api_trx_level
Removing this code in one commit makes it possible to easily restore
it, in case it turns out to be needed later.
A proper InnoDB shutdown after aborted startup was introduced
in commit 81b7fe9d38.
Also related to this is MDEV-11985, making read-only shutdown more robust.
If startup was aborted, there may exist recovered transactions that were
not rolled back. Relax the assertions accordingly.
Before killing the server, ensure that the redo log for the
incomplete transaction is flushed, so that the AUTO_INCREMENT
sequence will always be updated. Usually the INSERT
transaction would not have persisted the sequence before the
server was killed, but sometimes it could happen, causing
result mismatch.
Note: This test used to be called innodb_fts.innodb_fts_misc_debug.
Do not effectively set DEBUG_DBUG='d' by setting DEBUG_DBUG='-d,...'.
Instead, restore the saved value of DEBUG_DBUG.
Also, split the test innodb_fts.innodb_fts_misc_debug into
innodb_fts.crash_recovery and innodb_fts.misc_debug, and enable
these tests for --valgrind, the latter test for --embedded,
and the former tests for the non-debug server.
Encryption stores used key_version to
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION (offset 26)
field. Spatial indexes store RTREE Split Sequence Number
(FIL_RTREE_SPLIT_SEQ_NUM) in the same field. Both values
can't be stored in same field. Thus, current encryption
implementation does not support encrypting spatial indexes.
fil_space_encrypt(): Do not encrypt page if page type is
FIL_PAGE_RTREE (this is required for background
encryption innodb-encrypt-tables=ON).
create_table_info_t::check_table_options() Do not allow creating
table with ENCRYPTED=YES if table contains spatial index.
recv_log_format_0_recover(): Invoke log_decrypt_after_read() after
reading the old-format redo log buffer.
With this change, we will upgrade to an encrypted redo log that
is misleadingly carrying a MySQL 5.7.9 compatible format tag while
the log blocks (other than the header and the checkpoint blocks)
are in an incompatible, encrypted format.
That needs to be fixed by introducing a new redo log format tag that
indicates that the entire redo log is encrypted.
LOG_CHECKPOINT_ARRAY_END, LOG_CHECKPOINT_SIZE: Remove.
Change some error messages to refer to MariaDB 10.2.2 instead of
MySQL 5.7.9.
recv_find_max_checkpoint_0(): Do not abort when decrypting one of the
checkpoint pages fails.
If InnoDB is started in innodb_read_only mode such that
recovered incomplete transactions exist at startup
(but the redo logs are clean), an assertion will fail at shutdown,
because there would exist some non-prepared transactions.
logs_empty_and_mark_files_at_shutdown(): Do not wait for incomplete
transactions to finish if innodb_read_only or innodb_force_recovery>=3.
Wait for purge to finish in only one place.
trx_sys_close(): Relax the assertion that would fail first.
trx_free_prepared(): Also free recovered TRX_STATE_ACTIVE transactions
if innodb_read_only or innodb_force_recovery>=3.
Also, revert my earlier fix to MySQL 5.7 because this fix is more generic:
Bug#20874411 INNODB SHUTDOWN HANGS IF INNODB_FORCE_RECOVERY>=3
SKIPPED ANY ROLLBACK
trx_undo_fake_prepared(): Remove.
trx_sys_any_active_transactions(): Revert the changes.
Rewrite the test so that the main server is restarted, instead of
--exec $MYSQLD_CMD. In this way, the test can be run with Valgrind
and with any --mysqld=--innodb-page-size.
Also remove the workaround --skip-innodb-use-native-aio. It should
not be needed when we are inheriting the server parameters from
the test environment.
Datafile::validate_for_recovery(): Remove a redundant error message.
An error is already reported by Datafile::open_read_write() if the
file cannot be opened.
Also, do not assign SEARCH_ABORT, so that the full test will be executed
even if one step fails.
Test server startup with an empty encrypted redo log from 10.1.21.
FIXME: Pass the encryption parameters. Currently we only test startup
without properly set up encryption.
Remove the dependency on unzip. Instead, generate the InnoDB files
with perl.
log_block_checksum_is_ok(): Correct the error message.
recv_scan_log_recs(): Remove the duplicated error message for
log block checksum mismatch.
innobase_start_or_create_for_mysql(): If the server is in read-only
mode or if innodb_force_recovery>=3, do not try to modify the system
tablespace. (If the doublewrite buffer or the non-core system tables
do not exist, do not try to create them.)
innodb_shutdown(): Relax a debug assertion. If the system tablespace
did not contain a doublewrite buffer and if we started up in
innodb_read_only mode or with innodb_force_recovery>=3, it will not
be created.
dict_create_or_check_sys_tablespace(): Set the flag
srv_sys_tablespaces_open when the tables exist.
As noted in MDEV-11947, we should disable the InnoDB doublewrite buffer
during the tests, because when rewriting page checksums, innochecksum
would skip the pages that are in the doublewrite buffer area. Because
the doublewrite buffer is emptied on server startup and not shutdown,
we should initially start with the doublewrite buffer disabled, so that
there will be no warning messages for wrong page checksums in the
doublewrite buffer.
Also, correct the obvious typo where restart_options should have been
$restart_parameters, so that InnoDB is actually verifying that the
checksums were rewritten.
.. wsrep_max_ws_rows causes cluster to break when running
Galera cluster in TOI mode
Problem:
While copying records to temporary table during ALTER TABLE,
if there are more than wsrep_max_wsrep_rows records, the
command fails.
Fix:
Since, the temporary table records are not placed into the
binary log, wsrep_affected_rows must not be incremented.
Added a test.
Remove the debug parameter innodb_force_recovery_crash that was
introduced into MySQL 5.6 by me in WL#6494 which allowed InnoDB
to resize the redo log on startup.
Let innodb.log_file_size actually start up the server, but ensure
that the InnoDB storage engine refuses to start up in each of the
scenarios.
that must not send a response
Problem:- When using wsrep (w/ galera) and issuing commands that can
cause deadlocks, deadlock exception errors are sent in responses to
commands such as close prepared statement and close connection which,
by spec, must not send a response.
Solution:- In dispatch_command, we will handle COM_QUIT and COM_STMT_CLOSE
commands even in case of error.
Patch Credit:- Jaka Močnik
