Commit graph

18623 commits

Author SHA1 Message Date
Marko Mäkelä
8e36216a06 Import two ALTER TABLE…ALGORITHM=INPLACE tests from MySQL 5.6.
Also, revert part of MDEV-7685 that added an InnoDB abort when
ALTER TABLE…ALGORITHM=INPLACE is reporting that it ran out of
file space.
2017-04-05 14:46:35 +03:00
Marko Mäkelä
0d34dd7cfb MDEV-11840 InnoDB: "Cannot open <ib_buffer_pool file>" should not be an error
buf_load(): When the file cannot be opened for reading, issue a note,
not an error message.
2017-04-05 10:16:11 +03:00
Daniel Black
a7bb9e8fdb xtradb: fil_crypt_rotate_page, space_id should be compared to TRX_SYS_SPACE not space
like 9a218f4fb8 fil_crypt_rotate_page
 - space_id should be compared to TRX_SYS_SPACE not space

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-04-05 16:29:08 +10:00
Jan Lindström
85239bdfeb Merge pull request #350 from grooverdan/10.1-TRX_SYS_PAGE_NO
fil_crypt_rotate_page - space_id should be compared to TRX_SYS_SPACE not space
2017-04-05 08:40:47 +03:00
Sergei Petrunia
23b86a18e6 MariaRocks: temporarily disable 32-bit Windows builds 2017-04-04 17:46:56 +03:00
Sergei Petrunia
fb0b3640fe Merge branch '10.2' of github.com:MariaDB/server into bb-10.2-mariarocks 2017-04-04 17:12:06 +03:00
Sergei Petrunia
c85ea1ab6d MDEV-12439: (Temporary) Don't run MariaRocks tests under valgrind 2017-04-04 12:39:27 +03:00
Sergei Petrunia
d1fc3cc469 MariaRocks: temporarily disable a few failing tests 2017-04-04 12:36:14 +03:00
Daniel Black
9a218f4fb8 fil_crypt_rotate_page - space_id should be compared to TRX_SYS_SPACE not space
Fixes compile error that highlights problem:

/source/storage/innobase/fil/fil0crypt.cc: In function 'void fil_crypt_rotate_page(const key_state_t*, rotate_thread_t*)':
/source/storage/innobase/fil/fil0crypt.cc:1770:15: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
  if (space == TRX_SYS_SPACE && offset == TRX_SYS_PAGE_NO) {

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-04-04 15:47:21 +10:00
Marko Mäkelä
6e105d7535 Merge 10.1 into 10.2 2017-04-04 07:59:25 +03:00
Marko Mäkelä
9505c96839 MDEV-12428 SIGSEGV in buf_page_decrypt_after_read() during DDL
Also, some MDEV-11738/MDEV-11581 post-push fixes.

In MariaDB 10.1, there is no fil_space_t::is_being_truncated field,
and the predicates fil_space_t::stop_new_ops and fil_space_t::is_stopping()
are interchangeable. I requested the fil_space_t::is_stopping() to be added
in the review, but some added checks for fil_space_t::stop_new_ops were
not replaced with calls to fil_space_t::is_stopping().

buf_page_decrypt_after_read(): In this low-level I/O operation, we must
look up the tablespace if it exists, even though future I/O operations
have been blocked on it due to a pending DDL operation, such as DROP TABLE
or TRUNCATE TABLE or other table-rebuilding operations (ALTER, OPTIMIZE).
Pass a parameter to fil_space_acquire_low() telling that we are performing
a low-level I/O operation and the fil_space_t::is_stopping() status should
be ignored.
2017-04-03 22:09:28 +03:00
Vladislav Vaintroub
ff6f4d7db1 Windows : Fix compiling with VS2013
We do not use it now, but there is still no reason to break compilation
for other users.
2017-04-03 15:18:46 +00:00
Sergei Petrunia
5e0ed6912f Merge 10.2 into bb-10.2-mariarocks 2017-04-03 13:48:05 +03:00
Sergei Petrunia
5c66eb5c9f Disable compilation of storage/rocksdb/unittest/test_properties_collector
Met additional issues while trying to enable it.
2017-04-03 10:29:40 +03:00
Sergei Petrunia
44bc2a0ef7 Post-fixes for "Rename plugin rocksdb_se to rocksdb"
- Also rename plugin's config file
- And fix unit test's CMakeLists.txt
2017-04-02 23:38:28 +03:00
Sergei Petrunia
e7e6e30cb7 rocksdb_sys_vars should not be run if MyRocks is not compiled.
Copy the detection logic from 'rocksdb' test suite, add a note.
2017-04-02 23:14:12 +03:00
Vladislav Vaintroub
45a9470ff3 Rename plugin rocksdb_se to rocksdb 2017-04-02 18:22:22 +00:00
Vladislav Vaintroub
85a1d9edbe rocksdb - fix failing tests on Windows 2017-04-02 17:29:04 +00:00
Vladislav Vaintroub
3599b4989d MariaRocks : Fix looking up sst_dump 2017-04-02 17:29:04 +00:00
Vladislav Vaintroub
2be18d9b7f Fix the build on Windows.
Restore MYSQL_SYSVAR_UINT64_T and MYSQL_SYSVAR_SIZE_T in
plugin.h
2017-04-02 17:29:04 +00:00
Sergei Petrunia
c35a5884b0 Mark rocksdb.bulk_load and rocksdb.add_index_inplace_sstfilewriter as big
They use much more disk space than all other tests in rocksdb test suite
2017-04-02 14:51:59 +03:00
Sergei Petrunia
0aa056f642 Disable persistent_cache.test due to upstream MyRocks issue #579. 2017-04-02 13:12:19 +03:00
Sergei Petrunia
20d9fbcf9a MDEV-12424: binlog_encryption.encrypted_* tests fail with Can't locate autodie.pm error
Don't use Perl autodie module as there are platforms where it is not present
2017-04-02 12:18:23 +03:00
Sergei Petrunia
74889de426 Apply a workaround for MyRocks upstream issue #569
We are hitting this
https://github.com/facebook/mysql-5.6/issues/569
and we are not ready to merge the fix currently
2017-04-01 16:11:19 +03:00
Sergei Petrunia
788382f77a Fix coompilation on windows
Some Windows enviroments don't have uint64_t, and we dont really need
that type. Assume 64-bit integers take 8 bytes.
2017-04-01 06:32:20 +03:00
Sergei Golubchik
16a99c5ad9 MariaRocks tests: various cleanups
remove hard-coded paths (that assumed we're in a source tree)
remove various shell/perl/awk/whatsnot scripts, use mysqltest and perl
remove numerous --exec /some/unix/tool commands, use mysqltest and perl
2017-03-31 19:28:58 +02:00
Sergei Golubchik
d6d994bf42 remove two redundant *.inc files to restart a server
namely, restart_mysqld_with_option.inc and kill_and_restart_mysqld.inc -
use restart_mysqld.inc instead.

Also remove innodb_wl6501_crash_stripped.inc that wasn't used anywhere.
2017-03-31 19:28:58 +02:00
Sergei Golubchik
6e899642fe move rocksdb specific changes into rocksdb 2017-03-31 19:28:57 +02:00
Sergei Golubchik
9ce639af52 don't export all charsets to plugins
don't use internal server collation symbol names, use collation
properties and collation IDs, they are much more stable.
2017-03-31 19:22:20 +02:00
Sergei Golubchik
76a262cdf8 remove my_hash_const_element(), use Hash_set in C++ code 2017-03-31 19:22:19 +02:00
Sergei Golubchik
63798a6ea8 remove DB_TYPE_ROCKSDB 2017-03-31 19:22:19 +02:00
Marko Mäkelä
124bae082b MDEV-12289 Keep 128 persistent rollback segments for compatibility and performance
InnoDB divides the allocation of undo logs into rollback segments.
The DB_ROLL_PTR system column of clustered indexes can address up to
128 rollback segments (TRX_SYS_N_RSEGS). Originally, InnoDB only
created one rollback segment. In MySQL 5.5 or in the InnoDB Plugin
for MySQL 5.1, all 128 rollback segments were created.

MySQL 5.7 hard-codes the rollback segment IDs 1..32 for temporary undo logs.
On upgrade, unless a slow shutdown (innodb_fast_shutdown=0)
was performed on the old server instance, these rollback segments
could be in use by transactions that are in XA PREPARE state or
transactions that were left behind by a server kill followed by a
normal shutdown immediately after restart.

Persistent tables cannot refer to temporary undo logs or vice versa.
Therefore, we should keep two distinct sets of rollback segments:
one for persistent tables and another for temporary tables. In this way,
all 128 rollback segments will be available for both types of tables,
which could improve performance. Also, MariaDB 10.2 will remain more
compatible than MySQL 5.7 with data files from earlier versions of
MySQL or MariaDB.

trx_sys_t::temp_rsegs[TRX_SYS_N_RSEGS]: A new array of temporary
rollback segments. The trx_sys_t::rseg_array[TRX_SYS_N_RSEGS] will
be solely for persistent undo logs.

srv_tmp_undo_logs. Remove. Use the constant TRX_SYS_N_RSEGS.

srv_available_undo_logs: Change the type to ulong.

trx_rseg_get_on_id(): Remove. Instead, let the callers refer to
trx_sys directly.

trx_rseg_create(), trx_sysf_rseg_find_free(): Remove unneeded parameters.
These functions only deal with persistent undo logs.

trx_temp_rseg_create(): New function, to create all temporary rollback
segments at server startup.

trx_rseg_t::is_persistent(): Determine if the rollback segment is for
persistent tables.

trx_sys_is_noredo_rseg_slot(): Remove. The callers must know based on
context (such as table handle) whether the DB_ROLL_PTR is referring to
a persistent undo log.

trx_sys_create_rsegs(): Remove all parameters, which were always passed
as global variables. Instead, modify the global variables directly.

enum trx_rseg_type_t: Remove.

trx_t::get_temp_rseg(): A method to ensure that a temporary
rollback segment has been assigned for the transaction.

trx_t::assign_temp_rseg(): Replaces trx_assign_rseg().

trx_purge_free_segment(), trx_purge_truncate_rseg_history():
Remove the redundant variable noredo=false.
Temporary undo logs are discarded immediately at transaction commit
or rollback, not lazily by purge.

trx_purge_mark_undo_for_truncate(): Remove references to the
temporary rollback segments.

trx_purge_mark_undo_for_truncate(): Remove a check for temporary
rollback segments. Only the dedicated persistent undo log tablespaces
can be truncated.

trx_undo_get_undo_rec_low(), trx_undo_get_undo_rec(): Add the
parameter is_temp.

trx_rseg_mem_restore(): Split from trx_rseg_mem_create().
Initialize the undo log and the rollback segment from the file
data structures.

trx_sysf_get_n_rseg_slots(): Renamed from
trx_sysf_used_slots_for_redo_rseg(). Count the persistent
rollback segment headers that have been initialized.

trx_sys_close(): Also free trx_sys->temp_rsegs[].

get_next_redo_rseg(): Merged to trx_assign_rseg_low().

trx_assign_rseg_low(): Remove the parameters and access the
global variables directly. Revert to simple round-robin, now that
the whole trx_sys->rseg_array[] is for persistent undo log again.

get_next_noredo_rseg(): Moved to trx_t::assign_temp_rseg().

srv_undo_tablespaces_init(): Remove some parameters and use the
global variables directly. Clarify some error messages.

Adjust the test innodb.log_file. Apparently, before these changes,
InnoDB somehow ignored missing dedicated undo tablespace files that
are pointed by the TRX_SYS header page, possibly losing part of
essential transaction system state.
2017-03-31 18:53:04 +03:00
Sergei Petrunia
54a892e133 Post-merge fixes 2017-03-31 01:32:59 +03:00
Sergei Petrunia
5210c69e71 Merge 10.2 into bb-10.2-mariarocks 2017-03-31 01:14:00 +03:00
Marko Mäkelä
a0c79bcf00 Rename InnoDB transaction undo logging predicates.
trx::has_logged_persistent(): Renamed from trx_is_redo_rseg_updated().
Determines if a transaction has generated any persistent undo log.

trx::has_logged(): Renamed from trx_is_rseg_updated().
Determines if a transaction has generated any undo log.
2017-03-30 14:35:19 +03:00
Marko Mäkelä
d1374c5b77 MDEV-12345 post-merge fix. 2017-03-30 14:34:45 +03:00
Sergei Golubchik
da4d71d10d Merge branch '10.1' into 10.2 2017-03-30 12:48:42 +02:00
Sergei Golubchik
3a3b3d8ba8 cleanup: innodb files in cmake/
moved to storage/innobase/

also removed duplicate checks from storage/innobase/CMakeLists.txt
(they're all in storage/innobase/innodb.cmake)
2017-03-29 00:40:22 +02:00
Sergei Golubchik
92aafebd2a MDEV-11605 Assertion `(longlong) thd->status_var.local_memory_used >= 0 || !debug_assert_on_not_freed_memory' failed in my_malloc_size_cb_func
MyISAM in compute_vcols() - which is used only in mi_check code -
was computing indexed vcols into an internally allocated buffer
(not record[0]) and the buffer was calculated to be long enough to fit
every keyseg (a keyseg knows where its value in a record buffer is
and the length of the value).

This logic didn' work for prefix keys, because the keyseg length is the
length of a prefix, but the record buffer needs to fit the complete
value of a vcol. In this bug MyISAM was writing a 2K varchar
into a buffer too short.

Also it didn't work for repair-with-keycache, because that code
recalculats all vcols, not only indexed ones.

So, the buffer size (MYISAM_SHARE::vreclength) should include all
vcols' full lengths. But it was calculated in mi_open and low-level
MyISAM code has no knowledge of vcols.

As a fix we now recalculate MYISAM_SHARE::vreclength in
ha_myisam::setup_vcols_for_repair() which is always called
before compute_vcols().
2017-03-29 00:40:22 +02:00
Sergei Golubchik
1216244eb9 MDEV-11703 InnoDB background threads show up in the processlist
give threads more descriptive names
2017-03-29 00:40:21 +02:00
Sergei Golubchik
bf40e8069c MDEV-11059 don't build the server with jemalloc
don't build the server with jemalloc.
only build tokudb with it.
move tokudb into a separate package, because it brings a new dependency
2017-03-29 00:40:21 +02:00
Sergei Golubchik
31a5d7212f MDEV-12203 build fails, multiple errors, in xtradb/handler
disable XtraDB in a way that makes it impossible to be built
2017-03-29 00:40:21 +02:00
Vladislav Vaintroub
d3f82e3a67 Fix connect engine crashes in buildbot tests on Win64.
Do not truncate pointers returned by _findfirst to int.
2017-03-27 16:46:38 +00:00
Vicențiu Ciorbaru
7ebb81be1d Make mysql_ldb and sst_dump part of the rocksdb-engine component 2017-03-26 17:30:26 +03:00
Vicențiu Ciorbaru
3ade211a72 Do not build ldb binary as mysql_ldb does the same thing 2017-03-25 23:28:05 +02:00
Vicențiu Ciorbaru
86680e8b4f Skip rocksdb plugin if sources can not be fetched
Either we are building from a source package, in which case all sources
should be present, or we are building from a repository. The repository
needs to fetch the rocksdb submodule before building rocksdb.
2017-03-25 22:18:34 +02:00
Vladislav Vaintroub
d7c35a9992 Fix compiler error 2017-03-23 19:28:36 +00:00
Vladislav Vaintroub
e5b67a46bc MDEV-12345 Performance : replace calls to clock() inside trx_start_low() by THD::start_utime 2017-03-23 11:50:22 +00:00
Sergei Petrunia
4653b6e2a3 MDEV-12279: rocksdb.tbl_opt_data_index_dir fails, wrong error code
Change the returned error code to be ER_CANT_CREATE_TABLE.
Emit the warning text ourselves.

(When a query produces both an error and a warning, command-line client
with default settings will not provide any indication that the warning
is present, unfortunately. Need \W)
2017-03-23 11:06:26 +03:00
Sergei Golubchik
09a2107b1b Merge branch '10.0' into 10.1 2017-03-21 19:20:44 +01:00