Problem was two race condtion in Aria page cache:
- find_block() didn't inform free_block() that it had released requests
- free_block() didn't handle pinned blocks, which could happen if
free_block() was called as part of flush. This is fixed by not freeing
blocks that are pinned. This is safe as when maria_close() is called
when last thread is using a table, there can be no pinned blocks. For
other flush calls it's safe to ignore pinned blocks.
In MariaDB Server before 10.2, InnoDB will not be shut down properly
if startup fails. So, Valgrind failures are to be expected.
Disable the test under Valgrind. In 10.2, it should pass with Valgrind.
During merge of innodb code QC invalidation was removed from innodb part but not added to server part.
We decided to keep it in innodb to keep server/engine interface the same.
This only merges MDEV-12253, adapting it to MDEV-12602 which is already
present in 10.2 but not yet in the 10.1 revision that is being merged.
TODO: Error handling in crash recovery needs to be improved.
If a page cannot be decrypted (or read), we should cleanly abort
the startup. If innodb_force_recovery is specified, we should
ignore the problematic page and apply redo log to other pages.
Currently, the test encryption.innodb-redo-badkey randomly fails
like this (the last messages are from cmake -DWITH_ASAN):
2017-05-05 10:19:40 140037071685504 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1635994
2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE before MLOG_CHECKPOINT for tablespace 1
2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Data structure corruption
2017-05-05 10:19:41 140037071685504 [Note] InnoDB: Starting shutdown...
i=================================================================
==5226==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x612000018588 in thread T0
#0 0x736750 in operator delete(void*) (/mariadb/server/build/sql/mysqld+0x736750)
#1 0x1e4833f in LatchCounter::~LatchCounter() /mariadb/server/storage/innobase/include/sync0types.h:599:4
#2 0x1e480b8 in LatchMeta<LatchCounter>::~LatchMeta() /mariadb/server/storage/innobase/include/sync0types.h:786:17
#3 0x1e35509 in sync_latch_meta_destroy() /mariadb/server/storage/innobase/sync/sync0debug.cc:1622:3
#4 0x1e35314 in sync_check_close() /mariadb/server/storage/innobase/sync/sync0debug.cc:1839:2
#5 0x1dfdc18 in innodb_shutdown() /mariadb/server/storage/innobase/srv/srv0start.cc:2888:2
#6 0x197e5e6 in innobase_init(void*) /mariadb/server/storage/innobase/handler/ha_innodb.cc:4475:3
This patch corrects the fix for the bug mdev-10693.
It is critical for the function get_best_combination() not to call
create_ref_for_key() for constant tables.
This bug could manifest itself only in multi-table subqueries where
one of the tables is accessed by a constant primary key.
Port of mysql changeset by Georgi Kodinov <Georgi.Kodinov@Oracle.com>:
Bug #34325 : --add-drop-trigger option for mysqldump
Implemented the --add-drop-trigger option to prepend each
CREATE TRIGGER in the dump file with DROP TRIGGER.
The option is off by default. Added a test case.
The usage of windows functions when all tables were optimized away
by min/max optimization were not supported. As result a result,
the queries that used window functions with min/max aggregation
over the whole table returned wrong result sets.
The patch fixed this problem.
(from: http://buildbot.askmonty.org/buildbot/builders/p8-rhel6-bintar/builds/820/steps/test/logs/stdio)
Errors like the following indicate a potential endian storage issue:
rocksdb.rocksdb_range w1 [ fail ]
Test ended at 2017-04-27 18:56:11
CURRENT_TEST: rocksdb.rocksdb_range
--- /home/buildbot/maria-slave/p8-rhel6-bintar/build/storage/rocksdb/mysql-test/rocksdb/r/rocksdb_range.result 2017-04-27 17:41:27.740050347 -0400
+++ /home/buildbot/maria-slave/p8-rhel6-bintar/build/storage/rocksdb/mysql-test/rocksdb/r/rocksdb_range.reject 2017-04-27 18:56:11.230050346 -0400
@@ -25,15 +25,15 @@
select * from t2 force index (a) where a=0;
pk a b
0 0 0
-1 0 1
-2 0 2
-3 0 3
-4 0 4
-5 0 5
-6 0 6
-7 0 7
-8 0 8
-9 0 9
+16777216 0 1
+33554432 0 2
+50331648 0 3
+67108864 0 4
+83886080 0 5
+100663296 0 6
+117440512 0 7
+134217728 0 8
+150994944 0 9
# The rest are for code coverage:
explain
select * from t2 force index (a) where a=2;
@@ -41,23 +41,23 @@
1 SIMPLE t2 ref a a 4 const #
select * from t2 force index (a) where a=2;
pk a b
-20 2 20
-21 2 21
-22 2 22
-23 2 23
-24 2 24
-25 2 25
-26 2 26
-27 2 27
-28 2 28
-29 2 29
+335544320 2 20
+352321536 2 21
+369098752 2 22
+385875968 2 23
+402653184 2 24
+419430400 2 25
+436207616 2 26
+452984832 2 27
+469762048 2 28
+486539264 2 29
explain
select * from t2 force index (a) where a=3 and pk=33;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 const a a 8 const,const #
select * from t2 force index (a) where a=3 and pk=33;
pk a b
-33 3 33
+553648128 3 33
select * from t2 force index (a) where a=99 and pk=99;
pk a b
select * from t2 force index (a) where a=0 and pk=0;
...
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
This regression was caused by MDEV-12467 encryption.create_or_replace
hangs during DROP TABLE, where if table->is_stopping() (i.e. when
tablespace is dropped) background key rotation thread calls
fil_crypt_complete_rotate_space to release space and stop rotation.
However, that function does not decrease number of rotating
threads if table->is_stopping() is true.
- Update rocksdb submodule to revision
d616ebea23fa88cb9c2c8588533526a566d9cfab
- Normally this should be done by doing a merge from upstream
MyRocks, but now we are just updating rocksdb, MyRocks merge will
follow later.
- Apply a part of 4f6f072f6c74513087004755508eb6d6c432c5c1
use_direct_writes was renamed to use_direct_io_for_flush_and_compaction
- Update build_rocksdb.cmake - RocksDB have moved files around
innochecksum uses global variables. great, let's use them all the
way down, instead of passing them as arguments to innodb internals,
conditionally modifying function prototypes with #ifdefs
/home/travis/build/MariaDB/server/extra/mysqld_safe_helper.c:62:21: warning: too many arguments in call to 'do_usage'
do_usage(argv[0]);
~~~~~~~~ ^
1 warning generated.
/home/travis/build/MariaDB/server/sql/sql_class.cc:1941:15: warning: 'this' pointer cannot be null in well-defined C++ code; pointer may be assumed to always convert to true [-Wundefined-bool-conversion]
if (this && WSREP(this) && wsrep_thd_is_BF(this, FALSE))
^~~~ ~~
|| kv >= crypt_data->min_key_version,
encryption.innodb_encryption_tables failed in buildbot.
Now that key_version is not stored when page is read to
buf_page_t::key_version but always read from actual page
this assertion is not always valid.
The bug happened when the specification of a recursive CTE had
no recursive references at the top level of the specification.
In this case the regular processing of derived table references
of the select containing a non-recursive reference to this
recursive CTE misses handling the specification unit.
At the preparation stage any non-recursive reference to a
recursive CTE must be handled after the preparation of the
specification unit for this CTE. So we have to force this
preparation when regular handling of derived tables does not
do it.
This fixes a regression caused by MDEV-12428.
When we introduced a variant of fil_space_acquire() that could
increment space->n_pending_ops after space->stop_new_ops was set,
the logic of fil_check_pending_operations() was broken.
fil_space_t::n_pending_ios: A new field to track read or write
access from the buffer pool routines immediately before a block
write or after a block read in the file system.
fil_space_acquire_for_io(), fil_space_release_for_io(): Similar
to fil_space_acquire_silent() and fil_space_release(), but
modify fil_space_t::n_pending_ios instead of fil_space_t::n_pending_ops.
Adjust a number of places accordingly, and remove some redundant
tablespace lookups.
The following parts of this fix differ from the 10.2 version of this fix:
buf_page_get_corrupt(): Add a tablespace parameter.
In 10.2, we already had a two-phase process of freeing fil_space objects
(first, fil_space_detach(), then release fil_system->mutex, and finally
free the fil_space and fil_node objects).
fil_space_free_and_mutex_exit(): Renamed from fil_space_free().
Detach the tablespace from the fil_system cache, release the
fil_system->mutex, and then wait for space->n_pending_ios to reach 0,
to avoid accessing freed data in a concurrent thread.
During the wait, future calls to fil_space_acquire_for_io() will
not find this tablespace, and the count can only be decremented to 0,
at which point it is safe to free the objects.
fil_node_free_part1(), fil_node_free_part2(): Refactored from
fil_node_free().
This fixes a regression caused by MDEV-12428.
When we introduced a variant of fil_space_acquire() that could
increment space->n_pending_ops after space->stop_new_ops was set,
the logic of fil_check_pending_operations() was broken.
fil_space_t::n_pending_ios: A new field to track read or write
access from the buffer pool routines immediately before a block
write or after a block read in the file system.
fil_space_acquire_for_io(), fil_space_release_for_io(): Similar
to fil_space_acquire_silent() and fil_space_release(), but
modify fil_space_t::n_pending_ios instead of fil_space_t::n_pending_ops.
fil_space_free_low(): Wait for space->n_pending_ios to reach 0,
to avoid accessing freed data in a concurrent thread. Future
calls to fil_space_acquire_for_io() will not find this tablespace,
because it will already have been detached from fil_system.
Adjust a number of places accordingly, and remove some redundant
tablespace lookups.
FIXME: buf_page_check_corrupt() should take a tablespace from
fil_space_acquire_for_io() as a parameter. This will be done
in the 10.1 version of this patch and merged from there.
That depends on MDEV-12253, which has not been merged from 10.1 yet.
and make sure that private ca key is not deleted at the end of
the procedure, so that we could generate additional certificates
any time without regenerating everything
