Commit graph

187942 commits

Author SHA1 Message Date
Marko Mäkelä
42a4ae54c2 MDEV-21225 Remove ut_align() and use aligned_malloc()
Before commit 90c52e5291 introduced
aligned_malloc(), InnoDB always used a pattern of over-allocating
memory and invoking ut_align() to guarantee the desired alignment.

It is cleaner to invoke aligned_malloc() and aligned_free() directly.

ut_align(): Remove. In assertions, ut_align_down() can be used instead.
2019-12-05 06:42:31 +02:00
Marko Mäkelä
504202bd7f MDEV-21216: Remove fsp_header_get_space_id()
The function fsp_header_get_space_id() returns ulint instead of
uint32_t, only to be able to complain that the two adjacent
tablespace ID fields in the page differ. Remove the function,
and merge the check to the callers.

Also, make some more use of aligned_malloc().
2019-12-04 20:01:04 +02:00
Marko Mäkelä
2352f51625 MDEV-21174: Fix Galera
trx_rseg_write_wsrep_checkpoint(): Use mtr_t::OPT, because
much of the time, the redo log writes would be redundant.
This was broken in commit 56f6dab1d0.
2019-12-04 18:25:01 +02:00
Marko Mäkelä
6b5cdd4ff7 MDEV-19514: Update stale comments 2019-12-04 15:35:58 +02:00
Marko Mäkelä
95e903261e MDEV-21216 InnoDB does dirty read of TRX_SYS page before recovery
InnoDB startup was discovering undo tablespaces in a dirty way.
It was reading a possibly stale copy of the TRX_SYS page before
processing any redo log records.

srv_start(): Do not call buf_pool_invalidate(). Invoke
trx_rseg_get_n_undo_tablespaces() after the recovery has been initiated.

recv_recovery_from_checkpoint_start(): Assert that the buffer pool is
empty. This used to be guaranteed by the buf_pool_invalidate() call.

trx_rseg_get_n_undo_tablespaces(): Move to the calling compilation unit,
and reimplement in a simpler way.

srv_undo_tablespace_create(): Remove the constant parameter
size=SRV_UNDO_TABLESPACE_SIZE_IN_PAGES.

srv_undo_tablespace_open(): Reimplement in a cleaner way, with
more robust error handling.

srv_all_undo_tablespaces_open(): Split from srv_undo_tablespaces_init().

srv_undo_tablespaces_init(): Read all "undo001","undo002" tablespace
files directly, without consulting the TRX_SYS page via calling
trx_rseg_get_n_undo_tablespaces().

This is joint work with Thirunarayanan Balathandayuthapani.
2019-12-04 15:34:28 +02:00
Marko Mäkelä
e5dfdc5606 Cleanup: use constexpr for SRV_UNDO_TABLESPACE_SIZE_IN_PAGES 2019-12-04 15:34:28 +02:00
Vladislav Vaintroub
bf3034195f Part2: MDEV-12518 Unify sql_yacc.yy and sql_yacc_ora.yy
Generate sql_yacc_ora.yy from sql_yacc.yy , by using cmake script
which does some simple string(REPLACE)
2019-12-04 15:45:00 +04:00
Alexander Barkov
3f9d7072e9 Part1: MDEV-12518 Unify sql_yacc.yy and sql_yacc_ora.yy
Mixing %type and %expect declarations:
- sql_mode=ORACLE declarations look like an empty C code section
  inside sql_yacc.yy, consisting of an inactive #ifdef..#endif block.
- sql_mode=DEFAULT declarations look like an empty C code section
  inside sql_yacc_ora.yy, consisting of an inactive #ifdef..#endif block.

Mixing rules:
- Adding a special rule _empty to the shared rule section.
- Changing all instances of /*Empty*/ in sql_mode=DEFAULT and sql_mode=ORACLE
  specific sections to _empty.
- Changing the rest of C style comments /*xxx*/ in
  sql_mode=DEFAULT and sql_mode=ORACLE specific blocks to C++ style: //xxx
- Mixing sql_yacc.yy and sql_yacc_ora.yy, so
  sql_mode=ORACLE specific blocks sit in a comment inside sql_yacc.yy, and
  sql_mode=DEFAULT specific blocks sit in a comment inside sql_yacc_ora.yy.
2019-12-04 15:45:00 +04:00
Marko Mäkelä
bb45941685 MDEV-21205 Assertion failure in btr_sec_min_rec_mark
In commit af5947f433
the function btr_discard_page() is invoking btr_set_min_rec_mark()
with the wrong buf_block_t* object. node_ptr is on merge_block,
not block.

btr_discard_page(): Remove the variables merge_page, page, and
always refer to block->frame or merge_block->frame instead.
Also, limit the scope of node_ptr and avoid duplicated conditions.

btr_set_min_rec_mark(): Add a template parameter, so that the
caller can specify whether the page is supposed to have a left sibling.
Otherwise, the assertion (which was introduced in the same commit)
would fail in btr_discard_page().
2019-12-04 10:51:38 +02:00
Daniel Bartholomew
3b9a978a3b
bump the VERSION 2019-12-03 15:32:40 -05:00
Marko Mäkelä
2ac0e64cad MDEV-21174: Fix the 32-bit build
mtr_t::write(): Add explicit narrowing type casts to avoid warnings
about lossy implicit conversions.
2019-12-03 12:22:58 +02:00
Marko Mäkelä
af5947f433 MDEV-21174: Replace mlog_write_string() with mtr_t::memcpy()
mtr_t::memcpy(): Replaces mlog_write_string(), mlog_log_string().
The buf_block_t is passed a parameter, so that
mlog_write_initial_log_record_low() can be used instead of
mlog_write_initial_log_record_fast().

fil_space_crypt_t::write_page0(): Remove the fil_space_t* parameter.
2019-12-03 11:05:19 +02:00
Marko Mäkelä
87839258f8 MDEV-21174: Replace mlog_memset() with mtr_t::memset()
Passing buf_block_t helps us avoid calling
mlog_write_initial_log_record_fast() and page_get_page_no(),
and allows us to implement more debug checks, such as
that on ROW_FORMAT=COMPRESSED index pages, only the page header
may be modified by MLOG_MEMSET records.

fseg_n_reserved_pages(): Add a buf_block_t parameter.
2019-12-03 11:05:19 +02:00
Marko Mäkelä
caea64df18 Cleanup: Remove some page_get_page_no() calls
Refer to buf_page_t::id instead of parsing the tablespace identifier
or page number from the buffer pool page.
2019-12-03 11:05:19 +02:00
Marko Mäkelä
56f6dab1d0 MDEV-21174: Replace mlog_write_ulint() with mtr_t::write()
mtr_t::write(): Replaces mlog_write_ulint(), mlog_write_ull().
Optimize away writes if the page contents does not change,
except when a dummy write has been explicitly requested.

Because the member function template takes a block descriptor as a
parameter, it is possible to introduce better consistency checks.
Due to this, the code for handling file-based lists, undo logs
and user transactions was refactored to pass around buf_block_t.
2019-12-03 11:05:18 +02:00
Marko Mäkelä
504823bcce MDEV-21174: Cleanup MLOG_PAGE_CREATE
page_create_write_log(), mlog_write_initial_log_record():
Merge to the only caller, and use
mlog_write_initial_log_record_low() for writing the log record.
2019-12-03 11:05:18 +02:00
Marko Mäkelä
57444a3b30 MDEV-16264: Minor cleanup
aio_linux::m_max_io_count: Unused data member; remove.

aiocb::m_ret_len: Declare as the more compatible type size_t.
Unfortunately, ssize_t is not available on Microsoft Visual Studio.
2019-12-03 11:05:18 +02:00
Marko Mäkelä
cd92c6c83d MDEV-12353 preparation: Do not write MLOG_REC_MIN_MARK
btr_set_min_rec_mark(): Write MLOG_1BYTE instead of
MLOG_REC_MIN_MARK or MLOG_COMP_REC_MIN_MARK.

On ROW_FORMAT=COMPRESSED pages, the minimum record flag is not stored
at all. The flag is computed for the uncompressed page by
page_zip_decompress(). Hence, nothing needs to be logged for
ROW_FORMAT=COMPRESSED tables for this operation.

To facilitate crash-upgrade and hot backup from older versions,
we will retain the code to parse and apply the old log record types
MLOG_REC_MIN_MARK and MLOG_COMP_REC_MIN_MARK.
2019-12-03 11:05:18 +02:00
Marko Mäkelä
8ebd91c1a9 MDEV-12353 preparation: Do not write MLOG_FILE_WRITE_CRYPT_DATA
The MLOG_FILE_WRITE_CRYPT_DATA record was completely redundant.
It can be replaced with a single MLOG_WRITE_STRING record.

To facilitate upgrade from older versions, we will retain
fil_parse_write_crypt_data().

fil_crypt_parse(): Recover fil_space_crypt_t::write_page0().

fil_space_crypt_t::write_page0(): Write everything in a single
MLOG_WRITE_STRING for easy parsing.

fil_space_crypt_t::page0_offset: Remove.
2019-12-03 11:05:18 +02:00
Marko Mäkelä
bf2cc46798 MDEV-21133: Remove buf_frame_copy() 2019-12-03 11:05:18 +02:00
Aleksey Midenkov
6f89946892 MDEV-17554 versioning partition tests reorganize
Moved partition_rotation, partition_innodb to partition.test
2019-12-03 11:53:26 +03:00
Aleksey Midenkov
9ed8d364cd MDEV-17554 history partitioning cleanups
* Fixed missed warning on condition boundary
* REORGANIZE cases
* vers_utils.h removed
* test cases cleanup
2019-12-03 11:53:25 +03:00
Aleksey Midenkov
8ed646f071 Merge 10.4 into 10.5 2019-12-02 13:35:54 +03:00
Aleksey Midenkov
0b8b11b0b1 Merge 10.3 into 10.4 2019-12-02 12:51:53 +03:00
Aleksey Midenkov
1d46923a0f MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED (10.4)
Don't do skip_setup_conds() unless all errors are checked.

Fixes following errors:
      ER_PERIOD_NOT_FOUND
      ER_VERS_QUERY_IN_PARTITION
      ER_VERS_ENGINE_UNSUPPORTED
      ER_VERS_NOT_VERSIONED
2019-12-02 12:46:15 +03:00
Aleksey Midenkov
db32d9457e MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED
Don't do skip_setup_conds() unless all errors are checked.

Fixes following errors:
      ER_PERIOD_NOT_FOUND
      ER_VERS_QUERY_IN_PARTITION
      ER_VERS_ENGINE_UNSUPPORTED
      ER_VERS_NOT_VERSIONED
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
a7cf0db3d8 MDEV-21011 Table corruption reported for versioned partitioned table after DELETE: "Found a misplaced row"
LIMIT history partitions cannot be checked by existing algorithm of
check_misplaced_rows() because working history partition is
incremented each time another one is filled. The existing algorithm
gets record and tries to decide partition id for it by
get_partition_id(). For LIMIT history it will just get first
non-filled partition.

To fix such partitions it is required to do REBUILD instead of REPAIR.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
6dd41e008e MDEV-21155 Assertion with versioned table upon DELETE from view of view after replacing first view
When view is merged by DT_MERGE_FOR_INSERT it is then skipped from
processing and doesn't update WHERE clause with
vers_setup_conds(). Note that view itself cannot work in
vers_setup_conds() because it doesn't have row_start, row_end
fields. Thus it is required to descend down to material TABLE_LIST
through calls of mysql_derived_prepare() and run vers_setup_conds()
from there. Luckily, all views (views of views, views of views of
views, etc.) are linked in one list through next_global pointer, so we
can skip all views of views and get straight to non-view TABLE_LIST by
checking its merge_underlying_list property for zero value (it is
assigned by DT_MERGE_FOR_INSERT for merged derived tables).

We have to do that only for UPDATE and DELETE. Other DML commands
don't use WHERE clause.

MDEV-21146 Assertion `m_lock_type == 2' in handler::ha_drop_table upon LOAD DATA

LOAD DATA does not use WHERE and the above call of vers_setup_conds()
is not needed. unit->prepare() led to wrongly locked temporary table.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
97aa07abf5 MDEV-21147 Assertion `marked_for_read()' upon UPDATE on versioned table via view
"write set" for replication finally got its correct place
(mark_columns_per_binlog_row_image()). When done generally in
mark_columns_needed_for_update() it affects optimization
algorithm. used_key_is_modified, query_plan.using_io_buffer are
wrongly set and that leads to wrong prepare_for_keyread() which limits
read_set.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
498a96a478 MDEV-20441 ER_CRASHED_ON_USAGE upon update on versioned Aria table
Turn read cache off for update and multi-update for versioned
table. no_cache is reinited on each TABLE open because it is
applicable for specific algorithms.

As a side fix vers_insert_history_row() honors vers_write setting.

Aria with row_format=fixed uses IO_CACHE of type READ_CACHE for
sequential read in update loop. When history row is inserted inside
this loop the cache misses it and fails with error.

TODO:

Currently maria_extra() does not support SEQ_READ_APPEND. Probably it
might be possible to use this type of cache.
2019-12-02 11:48:37 +03:00
Alexander Barkov
523879dd51 A cleanup for MDEV-17088: disabling mariadb-conv-*.test for embedded 2019-11-30 13:18:28 +04:00
Kentoku
a3b63b8da3 MDEV-18973 CLIENT_FOUND_ROWS wrong in spider
Get count from last_used_con->info
Contributed by willhan at Tencent Games
Reviewed by Monty
2019-11-29 23:27:26 +09:00
Kentoku
e066723a41 MDEV-18973 CLIENT_FOUND_ROWS wrong in spider
Get count from last_used_con->info
Contributed by willhan at Tencent Games
2019-11-29 23:23:57 +09:00
Marko Mäkelä
326515878d MDEV-21133: Alignment hints for ROW_FORMAT=COMPRESSED
page_zip_compress(), page_zip_decompress_low(), page_zip_reorganize():
Make use of memcpy_aligned() and memset_aligned(), so that some
operations can be translated more efficiently.

No difference was observed for AMD64 on GCC 9.2.1. But, such hints could
enable more efficient code on less common instruction set architectures,
such as 32-bit ARM, POWER or MIPS.
2019-11-29 14:48:47 +02:00
Marko Mäkelä
5a00792c69 Merge 10.4 into 10.5 2019-11-29 11:25:40 +02:00
Marko Mäkelä
51a4260f00 MDEV-21133: Introduce memmove_aligned()
Both variants of the InnoDB page directory are aligned to the entry size
(16 bits). Inform the compiler about it.
2019-11-29 11:23:35 +02:00
Sergei Golubchik
57cab7cd51 fix ssl_crl test failures on newer OpenSSL
generalize the replacement
2019-11-29 09:08:52 +01:00
Sergei Golubchik
95898ac9bd Revert "Fix upgrade errors on eoan"
This reverts commit 0d345ec2e3.

Upgrades from 8.0 don't work yet, one has to dump/restore
manually to get the metadata out of the data dictionary.
2019-11-29 09:08:52 +01:00
Marko Mäkelä
7955e197d0 Cleanup: Remove a constant template parameter
dict_table_t::init_instant(): Remove bool replace_dropped=true.
The template parameter was unnecessarily added in
commit a044e326a8.
2019-11-29 08:52:14 +02:00
Jan Lindström
a8bf39cc3e Disable failing Galera test cases. 2019-11-29 08:23:15 +02:00
Tony Reix
ad5b7b157b MDEV-19510 Issue with: sql/wsrep_mysqld.cc : ip_len
Patch `36a2a185fe18` introduced `wsrep_server_incoming_address()` in `10.4`.
Since AIX `/usr/include/netinet/ip.h` header defines `ip_len` as `ip_ff.ip_flen`
and `size_t const ip_len` is preprocessed as `size_t const ip_ff.ip_vhltl.ip_x.ip_xlen`,
to prevent the define from overwriting code in MariaDB,
rename the variable name to `ip_len_mdb`.

This patch is done by Tony Reix <tony.reix@atos.net>.
This patch was submitted under MCA.
Closes #1307
2019-11-29 07:01:39 +01:00
Marko Mäkelä
0f71e9e66b Merge 10.4 into 10.5 2019-11-28 19:55:16 +02:00
Marko Mäkelä
3826178da8 Fix the Windows non-debug build
warning C4390:  ';': empty controlled statement found; is this the intent?
2019-11-28 19:53:00 +02:00
Sergei Golubchik
0d345ec2e3 Fix upgrade errors on eoan
mariadb packages conflict with mysql-8.0
2019-11-28 16:01:02 +01:00
Sergei Golubchik
dc7d0b5030 RPM packaging fixes for FC31
Obsoletes: cannot contain (x86-64) anymore

Python shebang must be specific
2019-11-28 16:01:02 +01:00
Marko Mäkelä
683bc53dc8 MDEV-21132: Add debug assertions
buf_flush_write_block_low(): Assert that the FIL_PAGE_LSN is set
and is not less than buf_page_t::oldest_modification.
2019-11-28 16:39:15 +02:00
Marko Mäkelä
beae2cf006 Merge 10.4 into 10.5 2019-11-28 16:35:20 +02:00
Marko Mäkelä
4beace3316 MDEV-21171 InnoDB is unnecessarily resetting FIL_PAGE_TYPE for full_crc32 files
Before commit c0f47a4a58 (MDEV-12026)
introduced the innodb_checksum_algorithm=full_crc32 format,
it was impossible to tell if InnoDB data files contained garbage in
the FIL_PAGE_TYPE header field (and possibly other fields).
This is because before commit 3926673ce7
in MySQL 5.1.48, InnoDB would write uninitialized data to some fields,
and because there was no way to tell with which InnoDB version a data
file was created.

If fil_space_t::full_crc32() holds, the data file cannot contain
uninitialized garbage or invalid FIL_PAGE_TYPE, and thus
fil_block_check_type() should not be invoked to correct anything.
2019-11-28 16:22:53 +02:00
Sergei Golubchik
576e85ad99 fix ssl_crl test failures on newer OpenSSL
generalize the replacement
2019-11-28 14:44:11 +01:00
Vladislav Vaintroub
bacdc4df61 MDEV-17088 - fix overlinking
mariadb-conv does not need to be linked with the client library
it does not talk to the server
2019-11-28 14:30:14 +01:00