Commit graph

187924 commits

Author SHA1 Message Date
Marko Mäkelä
8ebd91c1a9 MDEV-12353 preparation: Do not write MLOG_FILE_WRITE_CRYPT_DATA
The MLOG_FILE_WRITE_CRYPT_DATA record was completely redundant.
It can be replaced with a single MLOG_WRITE_STRING record.

To facilitate upgrade from older versions, we will retain
fil_parse_write_crypt_data().

fil_crypt_parse(): Recover fil_space_crypt_t::write_page0().

fil_space_crypt_t::write_page0(): Write everything in a single
MLOG_WRITE_STRING for easy parsing.

fil_space_crypt_t::page0_offset: Remove.
2019-12-03 11:05:18 +02:00
Marko Mäkelä
bf2cc46798 MDEV-21133: Remove buf_frame_copy() 2019-12-03 11:05:18 +02:00
Aleksey Midenkov
6f89946892 MDEV-17554 versioning partition tests reorganize
Moved partition_rotation, partition_innodb to partition.test
2019-12-03 11:53:26 +03:00
Aleksey Midenkov
9ed8d364cd MDEV-17554 history partitioning cleanups
* Fixed missed warning on condition boundary
* REORGANIZE cases
* vers_utils.h removed
* test cases cleanup
2019-12-03 11:53:25 +03:00
Aleksey Midenkov
8ed646f071 Merge 10.4 into 10.5 2019-12-02 13:35:54 +03:00
Aleksey Midenkov
0b8b11b0b1 Merge 10.3 into 10.4 2019-12-02 12:51:53 +03:00
Aleksey Midenkov
1d46923a0f MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED (10.4)
Don't do skip_setup_conds() unless all errors are checked.

Fixes following errors:
      ER_PERIOD_NOT_FOUND
      ER_VERS_QUERY_IN_PARTITION
      ER_VERS_ENGINE_UNSUPPORTED
      ER_VERS_NOT_VERSIONED
2019-12-02 12:46:15 +03:00
Aleksey Midenkov
db32d9457e MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED
Don't do skip_setup_conds() unless all errors are checked.

Fixes following errors:
      ER_PERIOD_NOT_FOUND
      ER_VERS_QUERY_IN_PARTITION
      ER_VERS_ENGINE_UNSUPPORTED
      ER_VERS_NOT_VERSIONED
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
a7cf0db3d8 MDEV-21011 Table corruption reported for versioned partitioned table after DELETE: "Found a misplaced row"
LIMIT history partitions cannot be checked by existing algorithm of
check_misplaced_rows() because working history partition is
incremented each time another one is filled. The existing algorithm
gets record and tries to decide partition id for it by
get_partition_id(). For LIMIT history it will just get first
non-filled partition.

To fix such partitions it is required to do REBUILD instead of REPAIR.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
6dd41e008e MDEV-21155 Assertion with versioned table upon DELETE from view of view after replacing first view
When view is merged by DT_MERGE_FOR_INSERT it is then skipped from
processing and doesn't update WHERE clause with
vers_setup_conds(). Note that view itself cannot work in
vers_setup_conds() because it doesn't have row_start, row_end
fields. Thus it is required to descend down to material TABLE_LIST
through calls of mysql_derived_prepare() and run vers_setup_conds()
from there. Luckily, all views (views of views, views of views of
views, etc.) are linked in one list through next_global pointer, so we
can skip all views of views and get straight to non-view TABLE_LIST by
checking its merge_underlying_list property for zero value (it is
assigned by DT_MERGE_FOR_INSERT for merged derived tables).

We have to do that only for UPDATE and DELETE. Other DML commands
don't use WHERE clause.

MDEV-21146 Assertion `m_lock_type == 2' in handler::ha_drop_table upon LOAD DATA

LOAD DATA does not use WHERE and the above call of vers_setup_conds()
is not needed. unit->prepare() led to wrongly locked temporary table.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
97aa07abf5 MDEV-21147 Assertion `marked_for_read()' upon UPDATE on versioned table via view
"write set" for replication finally got its correct place
(mark_columns_per_binlog_row_image()). When done generally in
mark_columns_needed_for_update() it affects optimization
algorithm. used_key_is_modified, query_plan.using_io_buffer are
wrongly set and that leads to wrong prepare_for_keyread() which limits
read_set.
2019-12-02 11:48:37 +03:00
Aleksey Midenkov
498a96a478 MDEV-20441 ER_CRASHED_ON_USAGE upon update on versioned Aria table
Turn read cache off for update and multi-update for versioned
table. no_cache is reinited on each TABLE open because it is
applicable for specific algorithms.

As a side fix vers_insert_history_row() honors vers_write setting.

Aria with row_format=fixed uses IO_CACHE of type READ_CACHE for
sequential read in update loop. When history row is inserted inside
this loop the cache misses it and fails with error.

TODO:

Currently maria_extra() does not support SEQ_READ_APPEND. Probably it
might be possible to use this type of cache.
2019-12-02 11:48:37 +03:00
Alexander Barkov
523879dd51 A cleanup for MDEV-17088: disabling mariadb-conv-*.test for embedded 2019-11-30 13:18:28 +04:00
Kentoku
a3b63b8da3 MDEV-18973 CLIENT_FOUND_ROWS wrong in spider
Get count from last_used_con->info
Contributed by willhan at Tencent Games
Reviewed by Monty
2019-11-29 23:27:26 +09:00
Kentoku
e066723a41 MDEV-18973 CLIENT_FOUND_ROWS wrong in spider
Get count from last_used_con->info
Contributed by willhan at Tencent Games
2019-11-29 23:23:57 +09:00
Marko Mäkelä
326515878d MDEV-21133: Alignment hints for ROW_FORMAT=COMPRESSED
page_zip_compress(), page_zip_decompress_low(), page_zip_reorganize():
Make use of memcpy_aligned() and memset_aligned(), so that some
operations can be translated more efficiently.

No difference was observed for AMD64 on GCC 9.2.1. But, such hints could
enable more efficient code on less common instruction set architectures,
such as 32-bit ARM, POWER or MIPS.
2019-11-29 14:48:47 +02:00
Marko Mäkelä
5a00792c69 Merge 10.4 into 10.5 2019-11-29 11:25:40 +02:00
Marko Mäkelä
51a4260f00 MDEV-21133: Introduce memmove_aligned()
Both variants of the InnoDB page directory are aligned to the entry size
(16 bits). Inform the compiler about it.
2019-11-29 11:23:35 +02:00
Sergei Golubchik
57cab7cd51 fix ssl_crl test failures on newer OpenSSL
generalize the replacement
2019-11-29 09:08:52 +01:00
Sergei Golubchik
95898ac9bd Revert "Fix upgrade errors on eoan"
This reverts commit 0d345ec2e3.

Upgrades from 8.0 don't work yet, one has to dump/restore
manually to get the metadata out of the data dictionary.
2019-11-29 09:08:52 +01:00
Marko Mäkelä
7955e197d0 Cleanup: Remove a constant template parameter
dict_table_t::init_instant(): Remove bool replace_dropped=true.
The template parameter was unnecessarily added in
commit a044e326a8.
2019-11-29 08:52:14 +02:00
Jan Lindström
a8bf39cc3e Disable failing Galera test cases. 2019-11-29 08:23:15 +02:00
Tony Reix
ad5b7b157b MDEV-19510 Issue with: sql/wsrep_mysqld.cc : ip_len
Patch `36a2a185fe18` introduced `wsrep_server_incoming_address()` in `10.4`.
Since AIX `/usr/include/netinet/ip.h` header defines `ip_len` as `ip_ff.ip_flen`
and `size_t const ip_len` is preprocessed as `size_t const ip_ff.ip_vhltl.ip_x.ip_xlen`,
to prevent the define from overwriting code in MariaDB,
rename the variable name to `ip_len_mdb`.

This patch is done by Tony Reix <tony.reix@atos.net>.
This patch was submitted under MCA.
Closes 
2019-11-29 07:01:39 +01:00
Marko Mäkelä
0f71e9e66b Merge 10.4 into 10.5 2019-11-28 19:55:16 +02:00
Marko Mäkelä
3826178da8 Fix the Windows non-debug build
warning C4390:  ';': empty controlled statement found; is this the intent?
2019-11-28 19:53:00 +02:00
Sergei Golubchik
0d345ec2e3 Fix upgrade errors on eoan
mariadb packages conflict with mysql-8.0
2019-11-28 16:01:02 +01:00
Sergei Golubchik
dc7d0b5030 RPM packaging fixes for FC31
Obsoletes: cannot contain (x86-64) anymore

Python shebang must be specific
2019-11-28 16:01:02 +01:00
Marko Mäkelä
683bc53dc8 MDEV-21132: Add debug assertions
buf_flush_write_block_low(): Assert that the FIL_PAGE_LSN is set
and is not less than buf_page_t::oldest_modification.
2019-11-28 16:39:15 +02:00
Marko Mäkelä
beae2cf006 Merge 10.4 into 10.5 2019-11-28 16:35:20 +02:00
Marko Mäkelä
4beace3316 MDEV-21171 InnoDB is unnecessarily resetting FIL_PAGE_TYPE for full_crc32 files
Before commit c0f47a4a58 (MDEV-12026)
introduced the innodb_checksum_algorithm=full_crc32 format,
it was impossible to tell if InnoDB data files contained garbage in
the FIL_PAGE_TYPE header field (and possibly other fields).
This is because before commit 3926673ce7
in MySQL 5.1.48, InnoDB would write uninitialized data to some fields,
and because there was no way to tell with which InnoDB version a data
file was created.

If fil_space_t::full_crc32() holds, the data file cannot contain
uninitialized garbage or invalid FIL_PAGE_TYPE, and thus
fil_block_check_type() should not be invoked to correct anything.
2019-11-28 16:22:53 +02:00
Sergei Golubchik
576e85ad99 fix ssl_crl test failures on newer OpenSSL
generalize the replacement
2019-11-28 14:44:11 +01:00
Vladislav Vaintroub
bacdc4df61 MDEV-17088 - fix overlinking
mariadb-conv does not need to be linked with the client library
it does not talk to the server
2019-11-28 14:30:14 +01:00
Alexander Barkov
4ad083cd64 MDEV-17088 Provide tools to encode/decode mysql-encoded file system names
The original patch was made by Takashi Sasaki <tsasaki609@gmail.com>.
2019-11-28 17:09:17 +04:00
Alexander E. Patrakov
f6003fbc8c MDEV-21140 Make galera_recovery.sh work with fs.protected_regular = 1 ()
The log file is opened as root since commit bb7a70c, so there is no need
to chown it.
2019-11-28 14:37:57 +02:00
Vladislav Vaintroub
ba95c303e3 MDEV-21167 LF_PINS::stack_ends_here inaccurate, leading to alloca() larger than stack
Use my_thread_var::stack_ends_here inside lf_pinbox_real_free() for address
where thread stack ends.

Remove LF_PINS::stack_ends_here.
It is not safe to assume that mysys_var that was used during pin allocation,
remains correct during free. E.g with binlog group commit in Innodb,
that frees pins for multiple Innodb transactions, it does not work
correctly.
2019-11-28 10:48:09 +01:00
Marko Mäkelä
a6e8a7df82 Cleanup: flst_read_addr(), fil_addr_t
fil_addr_t: Use exactly sized data types.

flst_read_addr(): Remove the unused parameter mtr.

page_offset(): Return uint16_t.
2019-11-28 11:44:40 +02:00
Vladislav Vaintroub
584ffa02f1 MDEV-19669 - fix matching CIDR address for proxy protocol.
Prior to this fix, when matching addresses using mask,
extra bits could be used for  comparison, e.g to
match with "a.b.c.d/24" , 27 bits were compared rather than 24.

The patch fixes the calculation.
2019-11-27 20:50:06 +01:00
Vladislav Vaintroub
29710b2839 Merge branch '10.4' into 10.5
# Conflicts:
#	win/packaging/extra.wxs.in
2019-11-27 16:02:34 +01:00
Vladislav Vaintroub
96c6b2b649 MDEV-19781 fixups
- It is not enough to have  and DeleteSymlinks actions
because within the same installation some exes executables can be both
added and removed, which needs both adding and removing some symlinks.

Instead of CreateSymlinks, there is now FixSymlinks function, which
goes through the list, and adds or removes the symlinks as needed.

- Implemented rollback for symlink custom action.

- Generate list of symlinks in C++ file, using CMake, rather than storing
lists as MSI properties.
2019-11-27 15:58:06 +01:00
Marko Mäkelä
3fc1f6260f MDEV-21158 trx_undo_seg_free() is never redo-logged
As part of commit 3c09f148f3
trx_undo_commit_cleanup() was always invoked with noredo=true.

The impact of this should be that some undo log pages may not be
correctly freed if the server is killed and crash recovery will be
performed. Similarly, if mariabackup --backup is being executed
concurrently with user transaction commits, it could happen that some
undo log pages in the backup will never be marked as free for reuse.

It seems that this bug should not have any user-visible
impact other than some undo pages being wasted.
2019-11-27 14:00:01 +02:00
Marko Mäkelä
ddbbf97670 Merge 10.4 into 10.5 2019-11-27 06:29:14 +02:00
Igor Babaev
ed355f59dd MDEV-20407 mysqld got signal 11; rowid filter
If a joined table is accessed by a full text index then now no range rowid
filter can be used when accessing the records of this table.

This bug was fixed by the patch for mdev-20056.
This commit adds only a test case with a query using a full text index
and a range condition for another index.
2019-11-26 13:38:19 -08:00
Marko Mäkelä
3eda03d0fe MDEV-21148: Assertion index->n_core_fields + n_add >= index->n_fields
Revert part of commit 6cedb671e9
because it turns out to be theoretically impossible to parse a
ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC metadata record where
the variable-length fields in the PRIMARY KEY have been written
as nonempty strings.
2019-11-26 20:46:25 +02:00
Monty
d1851b3009 Code cleanups
- Replace "class Repeat_count" with function call.
  There is no reason for this class to exists. More complex and more
  code than doing a function!  In addition the code didn't match what
  the val() function was doing.

Other things:
- Fixed compiler failure in print_cached_tables_callback()
2019-11-26 19:52:07 +02:00
Igor Babaev
4d4b2867a2 MDEV-20056 Assertion `!prebuilt->index->is_primary()' failed
in row_search_idx_cond_check

When usage of rowid filter is evaluated by the optimizer to join a table
to the current partial join employing a certain index it should be checked
that a key for at least the major component of this index can be constructed
using values from the columns of the partial join.
2019-11-26 08:31:21 -08:00
Rasmus Johansson
b6f7ec6a5b MDEV-19781 Create MariaDB named commands on Windows
Added CreateSymlinks and DeleteSymlinks functions to
CustomAction.cpp. Extra.wxs.in calls them.
2019-11-26 17:17:54 +02:00
Marko Mäkelä
dc75f3e06d MDEV-21152 Bogus debug assertion btr_pcur_is_after_last_in_tree() in ibuf code
As noted in commit abd45cdc38
a search with PAGE_CUR_GE may land on the supremum record on
a leaf page that is not the rightmost leaf page. This could occur
when all keys on the current page are smaller than the search key,
and the smallest key on the successor page is larger than the search key.

Hence, after a failed PAGE_CUR_GE search, assertions
btr_pcur_is_after_last_in_tree() are bogus
and should be replaced with btr_pcur_is_after_last_on_page().
2019-11-26 16:39:06 +02:00
Aleksey Midenkov
a35427f3fb MDEV-21127 Assertion in key_text::key_text()
Unsigned type while RHS expression could be less than 0.
2019-11-26 15:56:20 +03:00
Marko Mäkelä
25e2a556de MDEV-21133 Optimize access to InnoDB page header fields
Introduce memcpy_aligned<N>(), memcmp_aligned<N>(), memset_aligned<N>()
and use them for accessing InnoDB page header fields that are known
to be aligned.

MY_ASSUME_ALIGNED(): Wrapper for the GCC/clang __builtin_assume_aligned().
Nothing similar seems to exist in Microsoft Visual Studio, and the
C++20 std::assume_aligned is not available to us yet.

Explicitly specified alignment guarantees allow compilers to generate
faster code on platforms with strict alignment rules, instead of
emitting calls to potentially unaligned memcpy(), memcmp(), or memset().
2019-11-26 10:15:03 +02:00
Vladislav Vaintroub
86407a59b3 MDEV-16264 - Fix assertion `m_queue.empty() && !m_tasks_running' in tpool::task_group destructor
This particular assertion happened when shutting down Innodb IO.IO shutdown properly waits for all IOs to finish


However there is a race condition -
right after releasing last IO slot and before decrementing task count
in group, pending_io_count will be 0, but tasks_running will be 1,
leading to assertion.

The fix is to make task_group destructor to wait for last running task
to finish.
2019-11-25 17:10:01 +01:00