Commit graph

21387 commits

Author SHA1 Message Date
Marko Mäkelä
9ba0865b87 Merge 10.2 into 10.3 2019-04-08 21:38:13 +03:00
Marko Mäkelä
e7f426d2c9 MDEV-19212: Replace macros with type-safe inline functions
The regression that was reported in MDEV-19212 occurred due to use
of macros that did not ensure that the arguments have compatible
types.

ut_2pow_remainder(), ut_2pow_round(), ut_calc_align(): Define as
inline function templates.

UT_CALC_ALIGN(): Define as a macro, because this is used in
compile_time_assert(). Only starting with C++11 (MariaDB 10.4)
we could define the inline functions as constexpr.
2019-04-08 21:33:49 +03:00
Marko Mäkelä
f120a15b93 MDEV-19212 4GB Limit on large_pages - integer overflow
os_mem_alloc_large(): Invoke the macro ut_2pow_round() with the
correct argument type.

innobase_large_page_size, innobase_use_large_pages,
os_use_large_pages, os_large_page_size: Remove.
Simply refer to opt_large_page_size, my_use_large_pages.
2019-04-08 21:33:49 +03:00
Marko Mäkelä
4b822111ef MDEV-8139: Clean up the freeing of B-tree pages
btr_page_free(): Renamed from btr_page_free_low().
If scrubbing is enabled, zero out the page with proper redo logging.
Only pass ahi=true to fseg_free_page() if the page is actually indexed.

fil_space_t::modify_check(): Renamed from fsp_space_modify_check().

fsp_init_file_page(): Define inline.
2019-04-08 21:33:49 +03:00
Sergei Golubchik
7d720ca8de cmake: don't use generated files to detect a submodule
Even if Makefile for some reason was checked in in a submodule,
it is still a generated file, will be cleaned, won't be in a source
package. One cannot jump to conclusions if it doesn't exist.
2019-04-07 13:49:04 +02:00
Marko Mäkelä
7f5849a809 MDEV-18309: Remove unused code 2019-04-07 12:05:12 +03:00
Marko Mäkelä
cc492bfd4f Merge 10.2 into 10.3 2019-04-07 11:49:50 +03:00
Marko Mäkelä
867617a976 MDEV-18309: InnoDB reports bogus errors about missing #sql-*.ibd on startup
This is a follow-up to MDEV-18733. As part of that fix, we made
dict_check_sys_tables() skip tables that would be dropped by
row_mysql_drop_garbage_tables().

DICT_ERR_IGNORE_DROP: A new mode where the file should not be attempted
to be opened.

dict_load_tablespace(): Do not try to load the tablespace if
DICT_ERR_IGNORE_DROP has been specified.

row_mysql_drop_garbage_tables(): Pass the DICT_ERR_IGNORE_DROP mode.

fil_space_for_table_exists_in_mem(): Remove a parameter.
The only caller that passed print_error_if_does_not_exist=true
was row_drop_single_table_tablespace().
2019-04-07 10:57:38 +03:00
Marko Mäkelä
1d30b7b1d2 MDEV-12699 preparation: Clean up recv_sys
The recv_sys data structures are accessed not only from the thread
that executes InnoDB plugin initialization, but also from the
InnoDB I/O threads, which can invoke recv_recover_page().

Assert that sufficient concurrency control is in place.
Some code was accessing recv_sys data structures without
holding recv_sys->mutex.

recv_recover_page(bpage): Refactor the call from buf_page_io_complete()
into a separate function that performs necessary steps. The
main thread was unnecessarily releasing and reacquiring recv_sys->mutex.

recv_recover_page(block,mtr,recv_addr): Pass more parameters from
the caller. Avoid redundant lookups and computations. Eliminate some
redundant variables.

recv_get_fil_addr_struct(): Assert that recv_sys->mutex is being held.
That was not always the case!

recv_scan_log_recs(): Acquire recv_sys->mutex for the whole duration
of the function. (While we are scanning and buffering redo log records,
no pages can be read in.)

recv_read_in_area(): Properly protect access with recv_sys->mutex.

recv_apply_hashed_log_recs(): Check recv_addr->state only once,
and continuously hold recv_sys->mutex. The mutex will be released
and reacquired inside recv_recover_page() and recv_read_in_area(),
allowing concurrent processing by buf_page_io_complete() in I/O threads.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
aa3f7a107c MDEV-12699 preparation: Write MLOG_INDEX_LOAD for FTS_ tables
The record MLOG_INDEX_LOAD is supposed to be written to indicate that
some page modifications bypassed redo logging, and that redo logging
is now re-enabled. It was not written for fulltext indexes during
ALTER TABLE.

row_merge_write_redo(): Declare globally. Assert that the index
is neither a spatial nor fulltext index.

recv_mlog_index_load(): Observe a MLOG_INDEX_LOAD operation.

recv_parse_log_recs(): Handle MLOG_INDEX_LOAD also in multi-record
mini-transactions. Because of this omission, we should keep writing
MLOG_INDEX_LOAD in single-record mini-transactions, because older
versions of Mariabackup would fail.

row_fts_merge_insert(): Write MLOG_INDEX_LOAD for the auxiliary
tables of fulltext indexes.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
45d338dca8 MDEV-12699 preparation: Initialize the entire page on MLOG_ZIP_PAGE_COMPRESS
The record MLOG_ZIP_PAGE_COMPRESS is similar to MLOG_INIT_FILE_PAGE2
that it contains all the information needed to initialize the page.
Like for the other record, do initialize the entire page on recovery.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
1b95118c5f buf_page_get_gen(): Allow BUF_GET_IF_IN_POOL with a dummy page_size
The page_size argument to buf_page_get_gen() only matters when the
page is going to be loaded into the buffer pool. Allow callers to
pass a dummy parameter when using BUF_GET_IF_IN_POOL (which would
return NULL if the block is not in the buffer pool).
2019-04-06 21:25:43 +03:00
Marko Mäkelä
80f29211eb Fix a crash in CHECK TABLE for corrupted encrypted root page
btr_root_get(): Ignore the root->page.encrypted flag.
The purpose of this flag is questionable since
commit 8c43f96388.

btr_validate_index(): Avoid crash if btr_root_get() returns NULL.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
1d0380e029 MDEV-15528 preparation: Do not modify a freed page
btr_free_root(): Add the parameter bool invalidate.

btr_free_root_invalidate(): Remove.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
56df18be65 Clean up the parsing of MLOG_INIT_FILE_PAGE2
fsp_apply_init_file_page(): Renamed from fsp_init_file_page_low().

fsp_parse_init_file_page(): Remove. The redo log record has no
parameters.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
71f9552fd8 recv_recovery_is_on(): Add UNIV_UNLIKELY
Normally, InnoDB is not in the process of executing crash recovery.
Provide a hint to the compiler that the recovery-related code paths
are rarely executed.
2019-04-06 21:25:43 +03:00
Marko Mäkelä
d5a2bc6a0f Merge 10.2 into 10.3 2019-04-04 19:41:12 +03:00
Marko Mäkelä
f602385776 Do not pass table_name_t to printf-like functions 2019-04-04 08:57:53 +03:00
Marko Mäkelä
c676de1692 Fix the non-debug build 2019-04-03 21:00:13 +03:00
Marko Mäkelä
28636a92ed Merge 10.1 into 10.2 2019-04-03 19:58:47 +03:00
Marko Mäkelä
cad56fbaba MDEV-18733 MariaDB slow start after crash recovery
If InnoDB crash recovery was needed, the InnoDB function srv_start()
would invoke extra validation, reading something from every InnoDB
data file. This should be unnecessary now that MDEV-14717 made
RENAME operations crash-safe inside InnoDB (which can be
disabled in MariaDB 10.2 by setting innodb_safe_truncate=OFF).

dict_check_sys_tables(): Skip tables that would be dropped by
row_mysql_drop_garbage_tables(). Perform extra validation only
if innodb_safe_truncate=OFF, innodb_force_recovery=0 and
crash recovery was needed.

dict_load_table_one(): Validate the root page of the table.
In this way, we can deny access to corrupted or mismatching tables
not only after crash recovery, but also after a clean shutdown.
2019-04-03 19:56:03 +03:00
Marko Mäkelä
7984ea80de Remove a useless CHECK TABLE printout for debug builds 2019-04-03 19:56:03 +03:00
Marko Mäkelä
1f3bcff1f2 Remove unused declarations 2019-04-03 19:46:34 +03:00
Marko Mäkelä
fa14784301 Fix more -Wnonnull-compare
For some reason, GCC 8 did not issue warnings for all such comparisons.
2019-04-03 19:21:54 +03:00
Marko Mäkelä
a1ec7ac4f4 Clean up table_name_t
row_is_mysql_tmp_table_name(): Replaced with
dict_table_t::is_temporary_name() and table_name_t::is_temporary().

table_name_t: Add constructors.
2019-04-03 16:09:16 +03:00
Marko Mäkelä
532fffb4cc Fix the non-debug build 2019-04-03 14:07:18 +03:00
Marko Mäkelä
c6b8b05be4 Merge 10.2 into 10.3 2019-04-03 11:22:51 +03:00
Marko Mäkelä
03672a0573 MDEV-11487: Remove dict_table_get_n_sys_cols()
In MariaDB, InnoDB tables will always contain DATA_N_SYS_COLS = 3
columns, 2 or 3 of which are present in the clustered index.
We remove the predicate that was added in MySQL 5.7 as part of WL#7682.
2019-04-03 11:02:55 +03:00
Marko Mäkelä
dbc716675b Merge 10.1 into 10.2 2019-04-03 10:32:21 +03:00
Marko Mäkelä
c0fca2863b Fix -Wnonnull-compare
InnoDB and XtraDB had redundant assertions for checking that
function parameters that were declared as nonnull were not NULL.
2019-04-03 09:46:49 +03:00
Marko Mäkelä
977d7a7572 Merge 10.2 into 10.3 2019-04-03 08:21:43 +03:00
Sergei Golubchik
409f69cd74 cmake: only search for libraries that are needed
in particular, don't search for libjemalloc.a, which is only
needed for tokudb's ftcxx tests, when the tests aren't going
to be built.
2019-04-02 18:22:37 +02:00
Marko Mäkelä
e3f44d8d0e MDEV-19085: Remove a bogus debug assertion
MariaDB does support InnoDB tables with no stored columns.
(They are necessarily empty.)
2019-04-02 16:40:27 +03:00
Marko Mäkelä
44e64fd7e0 Merge 10.2 into 10.3 2019-04-02 13:48:42 +03:00
Marko Mäkelä
5633f83ca4 Fix integer type mismatch 2019-04-02 13:46:36 +03:00
Marko Mäkelä
8650848ec3 MDEV-19128 fil_name_parse() for MLOG_FILE_ is not portable
On Microsoft Windows, InnoDB writes the path separator \ to the
redo log file, while on all other platforms, / is being used.

fil_name_parse(): Normalize the parsed path separators to the
native format. This allows backups or data sets to be portable
between Windows and other systems.
2019-04-02 13:43:22 +03:00
Marko Mäkelä
7b42d892de Merge 10.2 into 10.3 2019-04-02 09:19:34 +03:00
Marko Mäkelä
bce380f2a5 Merge 10.1 into 10.2 2019-04-02 09:14:15 +03:00
Marko Mäkelä
b88f378648 Omit the definition of unused function yyset_extra()
This is follow-up for commit 619d22dde5
to fix the cmake -DWITH_EMBEDDED_SERVER build.
2019-04-02 08:50:53 +03:00
Marko Mäkelä
d59ad6972b MDEV-19085: Fix a typo that was caught by GCC 5.4 2019-04-01 18:13:11 +03:00
Marko Mäkelä
8f01a17292 Merge 10.2 into 10.3 2019-04-01 14:25:41 +03:00
Marko Mäkelä
f055da9b84 MDEV-19085 Assertion failures due to virtual columns after upgrading from 10.1
MariaDB before MDEV-5800 in version 10.2.2 did not support
indexed virtual columns. Non-persistent virtual columns were
hidden from storage engines. Only starting with MDEV-5800, InnoDB
would create internal metadata on virtual columns.

Similar to what was done in MDEV-18084, MDEV-18090, MDEV-18960, we adjust
one more code path for the old tables.

innobase_build_col_map(): Allocate space for virtual columns in col_map[]
but leave the entries at ULINT_UNDEFINED, noting that the virtual columns
were missing before the table was being rebuilt.
2019-04-01 14:24:15 +03:00
Marko Mäkelä
619d22dde5 Rebuild the InnoDB lexical analyzers with flex 2.6.4
InnoDB includes 3 parsers, which use 3 lexical analyzers that
are generated with flex. Flex versions before 2.6 emitted
the keyword "register", which is deprecated in C++17.

The lexical analyzers were regenerated as follows:

for s in storage/innobase storage/xtradb
do
	(cd "$s"/pars; ./make_flex.sh)
	touch "$s"/fts/*.l
	make -C "$s"/fts -f Makefile.query
done
2019-04-01 13:03:18 +03:00
Sergei Petrunia
833071b857 Disable tests in rocksdb_stress suite (which was enabled a few commits ago) 2019-04-01 12:58:51 +03:00
Marko Mäkelä
23eeecd68f MDEV-19111 Unused field INFORMATION_SCHEMA.INNODB_TABLESPACES_SCRUBBING.ROTATING_OR_FLUSHING
The MDEV-11738/MDEV-11581 fix was supposed to add the column
ROTATING_OR_FLUSHING to the INFORMATION_SCHEMA table
INNODB_TABLESPACES_ENCRYPTION, but it also added that column to
INNODB_TABLESPACES_SCRUBBING in InnoDB (not XtraDB).

The extra column was never initialized. We will remove it,
because key rotation has nothing to do with the scrubbing of
tablespace data.
2019-04-01 10:32:03 +03:00
Marko Mäkelä
0ffea01da5 Merge 10.2 into 10.3 2019-04-01 09:22:19 +03:00
Sergei Petrunia
fe1dfe3928 MDEV-19089, part : mark rocksdb.deadlock as "big test"
It turns out, moving it to rocksdb_stress test suite is not that easy.
2019-03-30 20:00:13 +03:00
Sergei Petrunia
c2d9a346ff MDEV-19089 part : adapt rocksdb_stress suite for MariaDB 2019-03-30 19:53:29 +03:00
Eugene Kosov
76934212eb remove unneeded code
rec_get_offsets() was previously called in a btr_cur_optimistic_update()
2019-03-29 23:42:26 +04:00
Eugene Kosov
b66164ab56 remove dead code 2019-03-29 23:40:28 +04:00