- Get the suite to work with dynamically-linked plugin (ha_rocksdb.so)
- Due to the push to keep everything MyRocks-related in storage/rocksdb,
there is no mysql-test/include/have_rocksdb.* anymore.
Make a copy of storage/rocksdb/mysql-test/rocksdb/include/have_rocksdb*,
hopefully these files wont be changed [often].
- Maria-fication of rocksdb_persistent_cache_path test.
This change should have been a part of
Merge 'merge-myrocks' into 'bb-10.2-mariarocks'
Merged cset:
Copy of
commit d1bb19b8f751875472211312c8e810143a7ba4b6
We probably should make submodule info a part of the mergetree process.
The function trx_purge_stop() was calling os_event_reset(purge_sys->event)
before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set()
call in srv_purge_coordinator_suspend() is protected by that X-latch.
It would seem a good idea to consistently protect both os_event_set()
and os_event_reset() calls with a common mutex or rw-lock in those
cases where os_event_set() and os_event_reset() are used
like condition variables, tied to changes of shared state.
For each os_event_t, we try to document the mutex or rw-lock that is
being used. For some events, frequent calls to os_event_set() seem to
try to avoid hangs. Some events are never waited for infinitely, only
timed waits, and os_event_set() is used for early termination of these
waits.
os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro
on other systems than Windows. TODO: remove this altogether and disable
innodb_use_native_aio on Windows.
os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.
log_write_flush_to_disk_low(): Invoke log_mutex_enter() at the end, to
avoid race conditions when changing the system state. (No potential
race condition existed before MySQL 5.7.)
Merged cset:
Copy of
commit d1bb19b8f751875472211312c8e810143a7ba4b6
Author: Manuel Ung <mung@fb.com>
Date: Fri Feb 3 11:50:34 2017 -0800
...
Add cardinality stats to information schema
Test suite parameters for 'rocksdb' test suite were disabled in order
to get mysqld to start at all when ha_rocksdb is a dynamic plugin.
A lot of tests depend on these parameters being enabled, though. Put
them back by using the loose- form.
Revert the MDEV-4396 tweak to innodb.innodb_bug14676111.
We must fix the root cause instead.
Allow gcol.innodb_virtual_purge to run on a non-debug build
(If wait_innodb_all_purged.inc is used in a non-debug test,
it will have no effect.)
Add the test innodb.index_merge_threshold from MySQL 5.7.
The test innodb.log_file_size_checkpoint was originally added to
MySQL 5.7 by me in a bug fix, to fix the interaction of WL#6494
(redo log resizing, introduced in MySQL 5.6) and WL#7142
(data file discovery based on MLOG_FILE_NAME records,
introduced in MySQL 5.7):
commit 70f9ef4e1220827132b50275ca7272f2bcca1864
Author: Marko Mäkelä <marko.makela@oracle.com>
Date: Wed May 21 13:31:29 2014 +0300
Bug#18755095 REDO LOG SIZE CHANGE AFTER CRASH RESULTS IN CHECKPOINT AGE
ERROR MESSAGE
This is a regression from fixing
Bug#18730524 REPEATED KILL+RESTART FAILS DUE TO MISSING MLOG_FILE_NAME
RECORD
innobase_start_or_create_for_mysql(): Invoke fil_names_clear() before
creating the "checkpoint" when changing redo log files.
Approved by Jimmy Yang on IM.
The relevant part of the test is that fil_names_clear() is invoked to
emit an MLOG_CHECKPOINT record before the redo log files are deleted.
In case the server is killed before ib_logfile0 has been deleted,
the old (not-yet-resized) redo log will be treated as valid. We do not
need to create a large number of tables for that.
I introduced the rec_printer object in MySQL to pretty-print raw InnoDB
records and index tuples in diagnostic messages. These objects are being
constructed unconditionally, even though the DBUG_PRINT is not enabled.
The unnecessary work is avoided by simply passing rec_printer(…).str()
to the DBUG_LOG macro that was introduced in MDEV-11713.
The issue was that JOIN::rollup_write_data() used
JOIN::tmp_table_param::[start_]recinfo, which had uninitialized data.
These fields have uninitialized data, because JOIN::tmp_table_param
currently only stores some grouping-related data fields. The data about
the work (temporary) tables themselves is stored in
join->join_tab[...].tmp_table_param.
The fix is to make JOIN::rollup_write_data follow this convention
and look at the right TMP_TABLE_PARAM object
The bug was not visible in current HEAD. Introduced test case to catch
regressions. Also improve error messages regarding distinct usage in
window functions.
The bug was caused by several issues.
2 problems in seek_io_cache. Due to wrong offsets used, we would end up
seeking way too much (first change), or over the intended seek point
(second change). Fixing it requires correctly detecting available data
in buffer (first change), and not using "IO_SIZE alligned" reads. The
second is needed because _my_b_cache_read adjusts the pos_in_file itself
based on read_pos and read_end. Pretending buffer is empty when we want
to force a read will aleviate this problem.
Secondly, the big-table cursors didn't repect the interface definitions
of always returning the rownumber that Table_read_cursor::fetch() would activate.
At the same time, next(), prev() and move_to() should not perform any
row activation.
Window functions need to be computed after applying the HAVING clause.
An optimization that we have for regular, non-window function, cases is
to apply having only during sending of the rows to the client. This
allows rows that should be filtered from the temporary table used to
store aggregation results to be stored there.
This behaviour is undesireable for window functions, as we have to
compute window functions on the result-set after HAVING is applied.
Storing extra rows in the table leads to wrong values as the frame
bounds might capture those -to be filtered afterwards- rows.
dict_init_free(): Make global, and move the call from
dict_close() to srv_free(), because this is initialized
earlier than dict_sys.
innobase_space_shutdown(): Do not leak srv_allow_writes_event.
Write only one encryption key to the checkpoint page.
Use 4 bytes of nonce. Encrypt more of each redo log block,
only skipping the 4-byte field LOG_BLOCK_HDR_NO which the
initialization vector is derived from.
Issue notes, not warning messages for rewriting the redo log files.
recv_recovery_from_checkpoint_finish(): Do not generate any redo log,
because we must avoid that before rewriting the redo log files, or
otherwise a crash during a redo log rewrite (removing or adding
encryption) may end up making the database unrecoverable.
Instead, do these tasks in innobase_start_or_create_for_mysql().
Issue a firm "Missing MLOG_CHECKPOINT" error message. Remove some
unreachable code and duplicated error messages for log corruption.
LOG_HEADER_FORMAT_ENCRYPTED: A flag for identifying an encrypted redo
log format.
log_group_t::is_encrypted(), log_t::is_encrypted(): Determine
if the redo log is in encrypted format.
recv_find_max_checkpoint(): Interpret LOG_HEADER_FORMAT_ENCRYPTED.
srv_prepare_to_delete_redo_log_files(): Display NOTE messages about
adding or removing encryption. Do not issue warnings for redo log
resizing any more.
innobase_start_or_create_for_mysql(): Rebuild the redo logs also when
the encryption changes.
innodb_log_checksums_func_update(): Always use the CRC-32C checksum
if innodb_encrypt_log. If needed, issue a warning
that innodb_encrypt_log implies innodb_log_checksums.
log_group_write_buf(): Compute the checksum on the encrypted
block contents, so that transmission errors or incomplete blocks can be
detected without decrypting.
Rewrite most of the redo log encryption code. Only remember one
encryption key at a time (but remember up to 5 when upgrading from the
MariaDB 10.1 format.)
The InnoDB redo log consists of a list of files that logically form
a bigger file, as if the individual files were concatenated together.
The first file will always be written on redo log checkpoint, because
the two checkpoint pages are at the start of the single logical
redo log file.
There is no technical reason why InnoDB requires at least 2 files
to exist. Let us reduce the minimum number to 1. In that way,
restoring from backups will become easier, since InnoDB can directly
deal with a single backed-up redo log file.
the test restarts the server, giving it 60 seconds to shutdown
and then killing it mercilessly.
make sure the server closes all MyISAM tables before shutdown,
as we cannot reliably expect it to make the deadline.
options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set
1. s/--log_slow_queries/--log-slow-queries/
2. disable log-slow-admin-statement/etc in mytr when doing mysqld --help
Change output of mysql_config --libs to have -lmariadb instead of
-lmysqlclient
Static linking is not yet supported, because static library has different
base name ( libmariadbclient.a static vs libmariadb.so shared)
Ever since MDEV-5800 enabled indexed virtual columns for InnoDB,
the InnoDB shutdown relied on close_connections() that would set
thd->killed for the InnoDB purge threads. Alas, the embedded server
shutdown is not invoking close_connections(), and thus InnoDB purge
threads fail to initiate shutdown, causing a hang.
innodb_inited: Remove. Use srv_was_started instead.
innobase_fast_shutdown: Remove. Use srv_fast_shutdown instead.
srv_running: Renamed from thd_destructor_myvar, and made global.
The value NULL means that shutdown was requested or the purge threads
should not be running because of innodb_read_only_mode=1.
innobase_init(): Set srv_was_started after ensuring that srv_running
was initialized. (In innodb_read_only mode, the purge threads are not
started and we do not care if srv_running==NULL.)
innobase_start_or_create_for_mysql(): Do not set srv_was_started.
Let it be set by the only caller innobase_init().
srv_purge_should_exit(): Check also srv_was_started and srv_running
when evaluating thd->killed.
PART 2 of the fix adds the logic of not using password column, unless it
exists. If password column is missing we attempt to use plugin &&
authentication_string columns.
PART 1 of the fix requires a bit of refactoring to not use hard-coded
field indices any more. Create classes that express the grant tables structure,
without exposing the underlying field indices.
Most of the code is converted to use these classes, except parts which
are not directly affected by the MDEV-11170. These however are TODO
items for subsequent refactoring.
The same approach is needed for LAST_VALUE, otherwise the LAST_VALUE sum
functions are not cleared correctly. Now LAST_VALUE behaves as NTH_VALUE
with 0 offset, only that the frame that it is examining is the bottom bound,
not the top bound.
The problematic queries involve unions. For unions we have an
optimization where we skip the ORDER BY clause in a query from one side
of the union if it will be performed later due to UNION.
EX:
(SELECT a from t1 ORDER BY a) ORDER BY b;
The first ordering by a is not necessary and it gets removed.
The problem is that we still need to resolve the Items before removing the
ORDER BY list from the
SELECT_LEX structure. During this final resolve step however, we forgot to
allow SET functions within the ORDER BY clause. This caused us to return
an "Invalid use of group function" error during the checking performed
by fix_fields in Item_sum::init_sum_func_check.
Fatal error: mysql.user table is damaged or in unsupported 3.20 format
The problem stems from MySQL 5.7.6. According to MySQL documentation:
In MySQL 5.7.6, the Password column was removed and all credentials are
stored in the authentication_string column.
If opening a MySQL 5.7.6 (and up) datadir with MariaDB 10.2, the user table
appears corrupted. In order to fix this, the server must be started with
--skip-grant-tables and then a subsequent mysql_upgrade command must be
issued.
This patch updates the mysql_upgrade command to also add the removed
Password column. The password column is necessary, otherwise
the mysql_upgrade script fails due to the Event_scheduler not being able
to start, as it can't find Event_priv in the table where it ought to be.
MySQL's version has column position 28 (0 index) vs our datadir version
expects position 29.
change the parser not to allow SERIAL as a normal data type.
make a special rule for it, where it could be used for define
fields, but not generated fields, not return type of a stored function, etc.
JOIN_CACHE's were initialized in check_join_cache_usage()
from make_join_readinfo(). After that make_join_readinfo() was looking
whether it's possible to use keyread. Later, after make_join_readinfo(),
optimizer decided whether to use filesort. And even later, at the
execution time, from join_read_first(), keyread was actually enabled.
The problem is, that if a query uses a vcol, base columns that it
depends on are automatically added to the read_set - because they're
needed to calculate the vcol. But if we're doing keyread, vcol is taken
from the index, not calculated, and base columns do not need to be
in the read set (even should not be - as they aren't getting values).
The bug was that JOIN_CACHE used read_set with base columns,
they were not read because of keyread, so it was caching garbage.
So read_set is only known after the keyread was decided. And after the
filesort was decided, as filesort doesn't use keyread. But
check_join_cache_usage() needs to be done in make_join_readinfo(),
as the code below depends on these checks,
Fix: keep JOIN_CACHE checks where they were, but move initialization
down to the very end of JOIN::optimize_inner. If keyread was enabled,
update the read_set to include only columns that are part of the index.
Copy the keyread logic from join_read_first() to happen at optimize time.
make init_read_record() to detect enabled keyread
and use index_* access methods, not rnd_*
this makes MariaDB to use keyread a lot more often than before
