Commit graph

18435 commits

Author SHA1 Message Date
Marko Mäkelä
3a50fd9d2f Simplify page_is_comp(). 2017-04-17 03:20:38 +03:00
Marko Mäkelä
a039d6c3e6 Remove unused tokens from the InnoDB SQL parser.
PARS_INTEGER_TOKEN: Remove. The lexer returns only PARS_INT_TOKEN.

PARS_FIXBINARY_LIT, PARS_BLOB_LIT: Remove. These are never returned
by the lexer. In sym_tab_add_bound_lit(), use PARS_STR_LIT.
2017-04-17 03:20:34 +03:00
Marko Mäkelä
e5eef05140 Minor code cleanup.
dict_index_is_sec_or_ibuf(): Use a single arithmetic expression.

rtr_split_page_move_rec_list(): Remove a redundant condition on
dict_index_is_sec_or_ibuf(). This function is always invoked on
a spatial index, which also is a secondary index.
2017-04-17 03:20:34 +03:00
Marko Mäkelä
bb81828938 Use page_is_leaf() where applicable. 2017-04-17 03:20:34 +03:00
Vladislav Vaintroub
8c9cd26c06 Rocksdb - disable tests that fail regularly on buildbot (MDEV-12474) 2017-04-07 17:00:51 +00:00
Vladislav Vaintroub
a33653eedb MDEV-12473 - fix rocksdb linking error
link rocksdb with librt, for clock_gettime()
2017-04-07 15:09:28 +00:00
Marko Mäkelä
85da56bf2d Remove the unused variable trx_t::support_xa.
Also, merge some test changes from MySQL 5.7.10, for deprecating
innodb_support_xa.
2017-04-07 13:33:59 +03:00
Vladislav Vaintroub
30cbbfbf77 MDEV-12452 postfix - use C style cast, not reinterpret_cast
to fix build on Win64
2017-04-07 06:09:25 +00:00
Vladislav Vaintroub
b64910ce27 MDEV-12452 MDEV-12453 : Fix building rocksdb and aws_key_management on macOS
use  CMAKE_CXX_STANDARD to set C++11 flags  with CMake 3.1+ (apples flags are somehow different from standard clang)
port htonbe16/32/64 macros for rocksdb
use reinterpret_cast<size_t> to cast macOS's pthread_t (pointer type) to  size_t , for rocksdb
2017-04-06 18:40:12 -04:00
Marko Mäkelä
1494147cf6 Merge 10.1 into 10.2 2017-04-06 09:52:25 +03:00
Marko Mäkelä
25d69ea012 MDEV-12198 innodb_defragment=1 crashes server on OPTIMIZE TABLE when FULLTEXT index exists
ha_innobase::defragment_table(): Skip corrupted indexes and
FULLTEXT INDEX. In InnoDB, FULLTEXT INDEX is implemented with
auxiliary tables. We will not defragment them on OPTIMIZE TABLE.
2017-04-06 08:56:25 +03:00
Marko Mäkelä
8d4871a953 Merge 10.0 into 10.1 2017-04-06 08:53:59 +03:00
Marko Mäkelä
8423294acf Make InnoDB doublewrite buffer creation more robust.
buf_dblwr_create(): Remove a bogus check for the buffer pool size.
Theoretically, there is no problem if the doublewrite buffer is
larger than the buffer pool. It could only cause trouble on crash
recovery, and on recovery the doublewrite buffer is read to a buffer
that is allocated outside of the buffer pool. Moreover, this check
was only performed when the database was initialized for the first
time.

On a normal startup, buf_dblwr_init() would not enforce any
rule on the innodb_buffer_pool_size.

Furthermore, in case of an error, commit the mini-transaction in order
to avoid an assertion failure on shutdown. Yes, this will leave the
doublewrite buffer in a corrupted stage, but the doublewrite buffer
should only be initialized when the data files are being initialized
from the scratch in the first place.
2017-04-05 16:24:44 +03:00
Marko Mäkelä
8e36216a06 Import two ALTER TABLE…ALGORITHM=INPLACE tests from MySQL 5.6.
Also, revert part of MDEV-7685 that added an InnoDB abort when
ALTER TABLE…ALGORITHM=INPLACE is reporting that it ran out of
file space.
2017-04-05 14:46:35 +03:00
Marko Mäkelä
0d34dd7cfb MDEV-11840 InnoDB: "Cannot open <ib_buffer_pool file>" should not be an error
buf_load(): When the file cannot be opened for reading, issue a note,
not an error message.
2017-04-05 10:16:11 +03:00
Daniel Black
a7bb9e8fdb xtradb: fil_crypt_rotate_page, space_id should be compared to TRX_SYS_SPACE not space
like 9a218f4fb8 fil_crypt_rotate_page
 - space_id should be compared to TRX_SYS_SPACE not space

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-04-05 16:29:08 +10:00
Jan Lindström
85239bdfeb Merge pull request #350 from grooverdan/10.1-TRX_SYS_PAGE_NO
fil_crypt_rotate_page - space_id should be compared to TRX_SYS_SPACE not space
2017-04-05 08:40:47 +03:00
Sergei Petrunia
23b86a18e6 MariaRocks: temporarily disable 32-bit Windows builds 2017-04-04 17:46:56 +03:00
Sergei Petrunia
fb0b3640fe Merge branch '10.2' of github.com:MariaDB/server into bb-10.2-mariarocks 2017-04-04 17:12:06 +03:00
Sergei Petrunia
c85ea1ab6d MDEV-12439: (Temporary) Don't run MariaRocks tests under valgrind 2017-04-04 12:39:27 +03:00
Sergei Petrunia
d1fc3cc469 MariaRocks: temporarily disable a few failing tests 2017-04-04 12:36:14 +03:00
Daniel Black
9a218f4fb8 fil_crypt_rotate_page - space_id should be compared to TRX_SYS_SPACE not space
Fixes compile error that highlights problem:

/source/storage/innobase/fil/fil0crypt.cc: In function 'void fil_crypt_rotate_page(const key_state_t*, rotate_thread_t*)':
/source/storage/innobase/fil/fil0crypt.cc:1770:15: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
  if (space == TRX_SYS_SPACE && offset == TRX_SYS_PAGE_NO) {

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2017-04-04 15:47:21 +10:00
Marko Mäkelä
6e105d7535 Merge 10.1 into 10.2 2017-04-04 07:59:25 +03:00
Marko Mäkelä
9505c96839 MDEV-12428 SIGSEGV in buf_page_decrypt_after_read() during DDL
Also, some MDEV-11738/MDEV-11581 post-push fixes.

In MariaDB 10.1, there is no fil_space_t::is_being_truncated field,
and the predicates fil_space_t::stop_new_ops and fil_space_t::is_stopping()
are interchangeable. I requested the fil_space_t::is_stopping() to be added
in the review, but some added checks for fil_space_t::stop_new_ops were
not replaced with calls to fil_space_t::is_stopping().

buf_page_decrypt_after_read(): In this low-level I/O operation, we must
look up the tablespace if it exists, even though future I/O operations
have been blocked on it due to a pending DDL operation, such as DROP TABLE
or TRUNCATE TABLE or other table-rebuilding operations (ALTER, OPTIMIZE).
Pass a parameter to fil_space_acquire_low() telling that we are performing
a low-level I/O operation and the fil_space_t::is_stopping() status should
be ignored.
2017-04-03 22:09:28 +03:00
Vladislav Vaintroub
ff6f4d7db1 Windows : Fix compiling with VS2013
We do not use it now, but there is still no reason to break compilation
for other users.
2017-04-03 15:18:46 +00:00
Sergei Petrunia
5e0ed6912f Merge 10.2 into bb-10.2-mariarocks 2017-04-03 13:48:05 +03:00
Sergei Petrunia
5c66eb5c9f Disable compilation of storage/rocksdb/unittest/test_properties_collector
Met additional issues while trying to enable it.
2017-04-03 10:29:40 +03:00
Sergei Petrunia
44bc2a0ef7 Post-fixes for "Rename plugin rocksdb_se to rocksdb"
- Also rename plugin's config file
- And fix unit test's CMakeLists.txt
2017-04-02 23:38:28 +03:00
Sergei Petrunia
e7e6e30cb7 rocksdb_sys_vars should not be run if MyRocks is not compiled.
Copy the detection logic from 'rocksdb' test suite, add a note.
2017-04-02 23:14:12 +03:00
Vladislav Vaintroub
45a9470ff3 Rename plugin rocksdb_se to rocksdb 2017-04-02 18:22:22 +00:00
Vladislav Vaintroub
85a1d9edbe rocksdb - fix failing tests on Windows 2017-04-02 17:29:04 +00:00
Vladislav Vaintroub
3599b4989d MariaRocks : Fix looking up sst_dump 2017-04-02 17:29:04 +00:00
Vladislav Vaintroub
2be18d9b7f Fix the build on Windows.
Restore MYSQL_SYSVAR_UINT64_T and MYSQL_SYSVAR_SIZE_T in
plugin.h
2017-04-02 17:29:04 +00:00
Sergei Petrunia
c35a5884b0 Mark rocksdb.bulk_load and rocksdb.add_index_inplace_sstfilewriter as big
They use much more disk space than all other tests in rocksdb test suite
2017-04-02 14:51:59 +03:00
Sergei Petrunia
0aa056f642 Disable persistent_cache.test due to upstream MyRocks issue #579. 2017-04-02 13:12:19 +03:00
Sergei Petrunia
20d9fbcf9a MDEV-12424: binlog_encryption.encrypted_* tests fail with Can't locate autodie.pm error
Don't use Perl autodie module as there are platforms where it is not present
2017-04-02 12:18:23 +03:00
Sergei Petrunia
74889de426 Apply a workaround for MyRocks upstream issue #569
We are hitting this
https://github.com/facebook/mysql-5.6/issues/569
and we are not ready to merge the fix currently
2017-04-01 16:11:19 +03:00
Sergei Petrunia
788382f77a Fix coompilation on windows
Some Windows enviroments don't have uint64_t, and we dont really need
that type. Assume 64-bit integers take 8 bytes.
2017-04-01 06:32:20 +03:00
Sergei Golubchik
16a99c5ad9 MariaRocks tests: various cleanups
remove hard-coded paths (that assumed we're in a source tree)
remove various shell/perl/awk/whatsnot scripts, use mysqltest and perl
remove numerous --exec /some/unix/tool commands, use mysqltest and perl
2017-03-31 19:28:58 +02:00
Sergei Golubchik
d6d994bf42 remove two redundant *.inc files to restart a server
namely, restart_mysqld_with_option.inc and kill_and_restart_mysqld.inc -
use restart_mysqld.inc instead.

Also remove innodb_wl6501_crash_stripped.inc that wasn't used anywhere.
2017-03-31 19:28:58 +02:00
Sergei Golubchik
6e899642fe move rocksdb specific changes into rocksdb 2017-03-31 19:28:57 +02:00
Sergei Golubchik
9ce639af52 don't export all charsets to plugins
don't use internal server collation symbol names, use collation
properties and collation IDs, they are much more stable.
2017-03-31 19:22:20 +02:00
Sergei Golubchik
76a262cdf8 remove my_hash_const_element(), use Hash_set in C++ code 2017-03-31 19:22:19 +02:00
Sergei Golubchik
63798a6ea8 remove DB_TYPE_ROCKSDB 2017-03-31 19:22:19 +02:00
Marko Mäkelä
124bae082b MDEV-12289 Keep 128 persistent rollback segments for compatibility and performance
InnoDB divides the allocation of undo logs into rollback segments.
The DB_ROLL_PTR system column of clustered indexes can address up to
128 rollback segments (TRX_SYS_N_RSEGS). Originally, InnoDB only
created one rollback segment. In MySQL 5.5 or in the InnoDB Plugin
for MySQL 5.1, all 128 rollback segments were created.

MySQL 5.7 hard-codes the rollback segment IDs 1..32 for temporary undo logs.
On upgrade, unless a slow shutdown (innodb_fast_shutdown=0)
was performed on the old server instance, these rollback segments
could be in use by transactions that are in XA PREPARE state or
transactions that were left behind by a server kill followed by a
normal shutdown immediately after restart.

Persistent tables cannot refer to temporary undo logs or vice versa.
Therefore, we should keep two distinct sets of rollback segments:
one for persistent tables and another for temporary tables. In this way,
all 128 rollback segments will be available for both types of tables,
which could improve performance. Also, MariaDB 10.2 will remain more
compatible than MySQL 5.7 with data files from earlier versions of
MySQL or MariaDB.

trx_sys_t::temp_rsegs[TRX_SYS_N_RSEGS]: A new array of temporary
rollback segments. The trx_sys_t::rseg_array[TRX_SYS_N_RSEGS] will
be solely for persistent undo logs.

srv_tmp_undo_logs. Remove. Use the constant TRX_SYS_N_RSEGS.

srv_available_undo_logs: Change the type to ulong.

trx_rseg_get_on_id(): Remove. Instead, let the callers refer to
trx_sys directly.

trx_rseg_create(), trx_sysf_rseg_find_free(): Remove unneeded parameters.
These functions only deal with persistent undo logs.

trx_temp_rseg_create(): New function, to create all temporary rollback
segments at server startup.

trx_rseg_t::is_persistent(): Determine if the rollback segment is for
persistent tables.

trx_sys_is_noredo_rseg_slot(): Remove. The callers must know based on
context (such as table handle) whether the DB_ROLL_PTR is referring to
a persistent undo log.

trx_sys_create_rsegs(): Remove all parameters, which were always passed
as global variables. Instead, modify the global variables directly.

enum trx_rseg_type_t: Remove.

trx_t::get_temp_rseg(): A method to ensure that a temporary
rollback segment has been assigned for the transaction.

trx_t::assign_temp_rseg(): Replaces trx_assign_rseg().

trx_purge_free_segment(), trx_purge_truncate_rseg_history():
Remove the redundant variable noredo=false.
Temporary undo logs are discarded immediately at transaction commit
or rollback, not lazily by purge.

trx_purge_mark_undo_for_truncate(): Remove references to the
temporary rollback segments.

trx_purge_mark_undo_for_truncate(): Remove a check for temporary
rollback segments. Only the dedicated persistent undo log tablespaces
can be truncated.

trx_undo_get_undo_rec_low(), trx_undo_get_undo_rec(): Add the
parameter is_temp.

trx_rseg_mem_restore(): Split from trx_rseg_mem_create().
Initialize the undo log and the rollback segment from the file
data structures.

trx_sysf_get_n_rseg_slots(): Renamed from
trx_sysf_used_slots_for_redo_rseg(). Count the persistent
rollback segment headers that have been initialized.

trx_sys_close(): Also free trx_sys->temp_rsegs[].

get_next_redo_rseg(): Merged to trx_assign_rseg_low().

trx_assign_rseg_low(): Remove the parameters and access the
global variables directly. Revert to simple round-robin, now that
the whole trx_sys->rseg_array[] is for persistent undo log again.

get_next_noredo_rseg(): Moved to trx_t::assign_temp_rseg().

srv_undo_tablespaces_init(): Remove some parameters and use the
global variables directly. Clarify some error messages.

Adjust the test innodb.log_file. Apparently, before these changes,
InnoDB somehow ignored missing dedicated undo tablespace files that
are pointed by the TRX_SYS header page, possibly losing part of
essential transaction system state.
2017-03-31 18:53:04 +03:00
Sergei Petrunia
54a892e133 Post-merge fixes 2017-03-31 01:32:59 +03:00
Sergei Petrunia
5210c69e71 Merge 10.2 into bb-10.2-mariarocks 2017-03-31 01:14:00 +03:00
Marko Mäkelä
a0c79bcf00 Rename InnoDB transaction undo logging predicates.
trx::has_logged_persistent(): Renamed from trx_is_redo_rseg_updated().
Determines if a transaction has generated any persistent undo log.

trx::has_logged(): Renamed from trx_is_rseg_updated().
Determines if a transaction has generated any undo log.
2017-03-30 14:35:19 +03:00
Marko Mäkelä
d1374c5b77 MDEV-12345 post-merge fix. 2017-03-30 14:34:45 +03:00
Sergei Golubchik
da4d71d10d Merge branch '10.1' into 10.2 2017-03-30 12:48:42 +02:00