This allows basic master crash-safety
- Un-comment and update relevant parts of the code
- Make rocksdb_rpl suite work like other MyRocks testsuites
(load the MyRocks plugin, don't start if it is not compiled in, etc)
- For now, disable all tests in the rocksdb_rpl suite.
- MariaDB-fication of rpl_rocksdb_2p_crash_recover test.
- Add include/index_merge*. Upstream has different files than MariaDB,
use copies theirs, not ours.
- There was a prblem with running "DDL-like" commands with binlog=ON:
MariaDB sets binlog_format=STATEMENT for the duration of such command
to prevent RBR replication from catching (and replicating) updates to
system tables.
However, MyRocks tries to prevent any writes to MyRocks tables with
binlog_format!=ROW.
- Added exceptions for DDL-type commands (ANALYZE TABLE, OPTIMIZE TABLE)
- Added special handling for "LOCK TABLE(s) myrocks_table WRITE".
Apply this patch from upstream:
commit 2c8deddfb67f1cd41ea3d1ac95aa1aa9327e3406
Author: Yoshinori Matsunobu <yoshinorim@users.noreply.github.com>
Date: Tue Aug 15 16:21:58 2017 -0700
Set exclusive_manual_compaction = false on manual compactions
Summary:
Combining exclusive manual compaction and
non-exclusive manual compaction may hit rocksdb assertion errors.
This diff makes all MyRocks internal manual compactions non exclusive.
Closes https://github.com/facebook/mysql-5.6/pull/682
Differential Revision: D5633619
Pulled By: yoshinorim
fbshipit-source-id: a90786d
The test mis-used MTR's "restart the server if it crashed or exited"
feature to try starting MyRocks plugin with invalid arguments.
Changed the test to use the --default-storage-engine=myisam which
allows the server to start when MyRocks fails to start.
This removes the need to "start the server with the arguments which
will caused it to fail to start", and so removes the race conditions
with MTR server restart code and mysqld.*.expect file.
It may produce test failures like this because of non-deterministic
cost calculations:
-1 SIMPLE t1 # col1 col1 259 NULL # Using where
+1 SIMPLE t1 # col1 NULL NULL NULL # Using where
- Fix the bad merge in drop_table.test
- Remove the obsolete rocksdb_info_log_level=info_level option
which caused warnings to be found in the error log.
commit 394d0712d3d46a87a8063e14e998e9c22336e3a6
Author: Anca Agape <anca@fb.com>
Date: Thu Jul 27 15:43:07 2017 -0700
Fix rpl.rpl_4threads_deadlock test broken by D5005670
Summary:
In D5005670 in fill_fields_processlist() function we introduced a point
where we were trying to take the LOCK_thd_data before the
synchronization point used by test
processlist_after_LOCK_thd_count_before_LOCK_thd_data. This was
happening in get_attached_srv_session() function called. Replaced this
with get_attached_srv_session_safe() and moved it after lock is aquired.
Reviewed By: tianx
Differential Revision: D5505992
fbshipit-source-id: bc53924
ha_partition creates temporary ha_XXX objects for its partitions when
performing DDL operations. The objects were created on a MEM_ROOT and
never deleted.
This works as long as ha_XXX objects free all data ha_XXX::close() and
don't rely on a proper destructor invocation. Unfortunately, ha_rocksdb
includes String members which need to be delete'd properly.
Fixed the bug by having ha_partition::~ha_partition delete these temporary
objects.
Disable memory leak check in debug server, if rocksdb is loaded.
There is some subtle bug somewhere in 3rd party code we cannot
do much about.
The bug is manifested as follows
Rocksdb does not shutdown worker threads, when plugin is shut down. Thus
OS does not unload the library since there are some active threads using
this library's code. Thus global destructors in the library do not run,
and there is still some memory allocated when server exits.
The workaround disables server's memory leak check, if rocksdb engine was
loaded.
(from: http://buildbot.askmonty.org/buildbot/builders/p8-rhel6-bintar/builds/820/steps/test/logs/stdio)
Errors like the following indicate a potential endian storage issue:
rocksdb.rocksdb_range w1 [ fail ]
Test ended at 2017-04-27 18:56:11
CURRENT_TEST: rocksdb.rocksdb_range
--- /home/buildbot/maria-slave/p8-rhel6-bintar/build/storage/rocksdb/mysql-test/rocksdb/r/rocksdb_range.result 2017-04-27 17:41:27.740050347 -0400
+++ /home/buildbot/maria-slave/p8-rhel6-bintar/build/storage/rocksdb/mysql-test/rocksdb/r/rocksdb_range.reject 2017-04-27 18:56:11.230050346 -0400
@@ -25,15 +25,15 @@
select * from t2 force index (a) where a=0;
pk a b
0 0 0
-1 0 1
-2 0 2
-3 0 3
-4 0 4
-5 0 5
-6 0 6
-7 0 7
-8 0 8
-9 0 9
+16777216 0 1
+33554432 0 2
+50331648 0 3
+67108864 0 4
+83886080 0 5
+100663296 0 6
+117440512 0 7
+134217728 0 8
+150994944 0 9
# The rest are for code coverage:
explain
select * from t2 force index (a) where a=2;
@@ -41,23 +41,23 @@
1 SIMPLE t2 ref a a 4 const #
select * from t2 force index (a) where a=2;
pk a b
-20 2 20
-21 2 21
-22 2 22
-23 2 23
-24 2 24
-25 2 25
-26 2 26
-27 2 27
-28 2 28
-29 2 29
+335544320 2 20
+352321536 2 21
+369098752 2 22
+385875968 2 23
+402653184 2 24
+419430400 2 25
+436207616 2 26
+452984832 2 27
+469762048 2 28
+486539264 2 29
explain
select * from t2 force index (a) where a=3 and pk=33;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 const a a 8 const,const #
select * from t2 force index (a) where a=3 and pk=33;
pk a b
-33 3 33
+553648128 3 33
select * from t2 force index (a) where a=99 and pk=99;
pk a b
select * from t2 force index (a) where a=0 and pk=0;
...
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
- Update rocksdb submodule to revision
d616ebea23fa88cb9c2c8588533526a566d9cfab
- Normally this should be done by doing a merge from upstream
MyRocks, but now we are just updating rocksdb, MyRocks merge will
follow later.
- Apply a part of 4f6f072f6c74513087004755508eb6d6c432c5c1
use_direct_writes was renamed to use_direct_io_for_flush_and_compaction
- Update build_rocksdb.cmake - RocksDB have moved files around
