Commit graph

188369 commits

Author SHA1 Message Date
Alexey Botchkov
9dadfdcde5 MDEV-14024 PCRE2.
Related changes in the server code.
2019-12-21 10:34:02 +01:00
Sergei Golubchik
aade6e53d3 fix a bad merge
in 10.1+ one should use

MY_CHECK_AND_SET_COMPILER_FLAG("-Wno-address-of-packed-member")

and it's already done in storage/tokudb/PerconaFT/CMakeLists.txt
2019-12-20 19:47:31 +01:00
Marko Mäkelä
8174e68895 MDEV-21371 Assertion failure in page_rec_get_next_low() during innodb_gis.rtree_compress
A debug assertion that was added in
commit ed0793e096
turns out to be too strict. In the test innodb_gis.rtree_compress,4k
the function is sometimes being invoked by purge for a
spatial index root page that is not a leaf page (PAGE_LEVEL is 1).
2019-12-20 19:21:37 +02:00
Marko Mäkelä
ce70573f62 MDEV-21353 Assertion failures due to btr_pcur_restore_pos() misbehaving
In commit befde6e97e
a bogus condition was added, aiming to avoid debug assertion failures
during ALTER TABLE...IMPORT TABLESPACE.

page_cur_delete_rec(): Remove the bogus condition, and replace the
use of buf_block_modify_clock_inc() with a lower-level operation
that skips the debug checks that would fail during IMPORT.
2019-12-20 17:57:32 +02:00
Eugene Kosov
f6b58d2916 MDEV-21239 ASAN use-after-poison in a server shutdown in innodb.innodb_buffer_pool_resize
If we manually poison memory region, ASAN requires that we manually unpoison
it too. We didn't do it and munmap() didn't do it too. That was the issue.

os_mem_free_large(): unpoison memory region. Just in case.

https://github.com/google/sanitizers/issues/182
86acaa9457/compiler-rt/include/sanitizer/asan_interface.h (L21)
2019-12-20 19:59:58 +08:00
Alexander Barkov
5e0c80c2a5 An extra test with NULLIF() for MDEV-13995 MAX(timestamp) returns a wrong result near DST change 2019-12-20 15:59:09 +04:00
Eugene Kosov
305081a735 check I/O buffer size and alignment in InnoDB 2019-12-19 23:46:49 +08:00
Alexander Barkov
1f1e3ce8a1 MDEV-21319 COUNT(*) returns 1, actual SELECT returns no result in 10.3.21, but 1 result in 10.1.41
Item_ref::val_(datetime|time)_packed() erroneously called
(*ref)->val_(datetime|time)_packed().

- Fixing to call (*ref)->val_(datetime|time)_packed_result().
- Backporting Item::val_(datetime|time)_packed_result() from 10.3.
- Fixing Item_field::get_date_result() to handle null_value in the same
  way how Item_field::get_date() does.
2019-12-19 14:03:54 +04:00
Eugene Kosov
a5a433e256 fix windows compilation 2019-12-18 23:35:33 +08:00
Marko Mäkelä
44be8652c5 Cleanup: Remove fil_space_get_flags()
Replace fil_space_get_flags(space) == ULINT_UNDEFINED
with the functionally equivalent fil_space_get_size(space) == 0.
2019-12-18 16:27:26 +02:00
Eugene Kosov
3d3d8f1012 MDEV-21337 fix aligned_malloc()
do not fallback to malloc(), always return properly aligned buffer
2019-12-18 20:09:52 +08:00
Sergei Petrunia
984b3c1544 MDEV-21341: Fix UBSAN failures, part #3
One may not call memcpy(dst, src=NULL, size), even if size==0.
2019-12-18 13:11:07 +03:00
Sergei Petrunia
8b9db11718 MDEV-21341: Fix optimizer-related UBSAN failures, part #2
Remove Query_tables_list::lock_tables_state - it is not used and it causes
errors like this:

sql_lex.h:1675:7: runtime error: load of value 2779096485, which is not a
valid value for type 'enum_lock_tables_state'
2019-12-18 12:56:54 +03:00
Sergei Petrunia
0ac0bc8cb6 MDEV-21341: Fix optimizer-related UBSAN failures, part #1:
Fix wrong typecast
2019-12-18 12:34:14 +03:00
Jan Lindström
c3824766c5 Fortify galera_partition test. 2019-12-18 10:02:57 +02:00
Jan Lindström
e9259d50f0 MDEV-21335 : Galera test failure on suite wsrep
Problem was that wsrep_on was OFF.
2019-12-18 10:02:57 +02:00
Jan Lindström
088de81d96 MDEV-21335 : Galera test failure on suite wsrep
Problem was that wsrep_on was OFF.

This is 10.4 version.
2019-12-18 08:22:07 +02:00
Vladislav Vaintroub
80c97f8c0c MDEV-21343 Threadpool/Unix- wait_begin() function does not wake/create threads, when it should
Fixed the condition for waking up/creating another thread.

If there is some work to do (if the request queue is not empty),
a thread should be woken or created.

The condition was incorrect since 18c9b34
2019-12-17 22:39:06 +01:00
Vladislav Vaintroub
189fa30085 MDEV-21343 Threadpool/Unix- wait_begin() function does not wake/create threads, when it should
Fixed the condition for waking up/creating another thread.

If there is some work to do (if the request queue is not empty),
a thread should be woken or created.

The condition was incorrect since 18c9b345b4
2019-12-17 21:57:40 +01:00
Sergei Golubchik
0d70f40bdb Disable -Werror for rocksdb submodule
We cannot have -Werror for third-party submodules because
whenever they break the build we cannot fix it as needed.
2019-12-17 21:22:57 +01:00
Varun Gupta
164cf4f463 MDEV-16579: Wrong result of query using DISTINCT COUNT(*) OVER (*)
The query requires 2 temporary tables for execution, the window function
is always attached to the last temporary table, but in this case the
result field of the window function points to the first temporary table
rather than the last one.
Fixed this by not changing window function items with temporary table
items of the first temporary table.
2019-12-17 21:45:53 +05:30
Marko Mäkelä
fb4a897fd9 MDEV-12353 preparation: Remove UNIV_LOG_LSN_DEBUG
The debug instrumentation with the MLOG_LSN pseudo-record has not been
used for debugging for years. Let us remove this code now.
It would have to be removed as part of MDEV-12353 or MDEV-14425 anyway,
when implementing a new redo log file format.
2019-12-17 15:39:21 +02:00
Marko Mäkelä
3c7718150d Merge 10.4 into 10.5 2019-12-17 14:46:57 +02:00
Marko Mäkelä
59a088744d Remove unused mlog_catenate_ulint_compressed()
The function was only used by trx_undo_page_init_log()
(writing the MLOG_UNDO_INIT record), which was removed in
commit ccb3550221.
2019-12-16 13:40:00 +02:00
Marko Mäkelä
c24253d0fa MDEV-21174: Fix the WITH_INNODB_EXTRA_DEBUG build
One reference to PageConverter::m_page_zip_ptr was not adjusted
commit 745fd4b39f when the
data member was removed.
2019-12-16 13:14:05 +02:00
Alexander Barkov
3d98892232 Merge remote-tracking branch 'origin/5.5' into 10.1 2019-12-16 13:08:17 +04:00
Alexander Barkov
fc860d3fa3 MDEV-21065 UNIQUE constraint causes a query with string comparison to omit a row in the result set 2019-12-16 12:57:08 +04:00
Marko Mäkelä
89633995e4 MDEV-16678: Actually ignore #sql-ib tables
Apparently, regular expression operations that remove entire lines
of output do not work with list_files, and hence the adjustments in
commit 1c282d4bc4 were ineffective.

For cat_file (preceded by list_files_write_file) the replace_regex
does work.

For some reason, for suite/parts/inc/partition_crash_exchange.inc
some file names will be lost when using list_files_write_file
instead of list_files.

We use a precise pattern match. dict_mem_create_temporary_tablename()
is generating #sql-ib names followed by decimal digits only.
2019-12-16 08:58:01 +02:00
Teemu Ollakka
67e063eb94 Update wsrep-lib. (#1426)
This commit updates the wsrep-lib. The changes are a cleanup in
client_state TOI processing and stub methods for future extensions.
2019-12-16 07:50:15 +02:00
Marko Mäkelä
28c89b7151 Merge 10.4 into 10.5 2019-12-16 07:47:17 +02:00
Vicențiu Ciorbaru
cf0823f9b5 Update test result post merge
Columns order is now according to standard SQL
2019-12-13 20:52:44 +02:00
Marko Mäkelä
745fd4b39f MDEV-21174: Remove some mlog_write_initial_log_record_fast()
Pass buf_block_t* to more functions that write redo log.

page_zip_write_node_ptr(), page_zip_write_blob_ptr(),
page_zip_compress_write_log_no_data():
Take buf_block_t* as parameter, and do not tolerate mtr=NULL.

page_zip_compress(): Do not tolerate mtr=NULL.

page_zip_dir_insert(): Take page_cur_t* as parameter.

mlog_write_initial_log_record(): Remove. This function was unused.

RecIterator::remove(): Remove the redundant page_zip parameter.

PageConverter::m_page_zip_ptr: Remove.
2019-12-13 18:15:51 +02:00
Marko Mäkelä
2b5a269cb4 MDEV-21174: Clean up record insertion
page_cur_insert_rec_low(): Take page_cur_t* as a parameter,
and do not tolerate mtr=NULL.

page_cur_insert_rec_zip(): Do not tolerate mtr=NULL.
2019-12-13 18:15:51 +02:00
Marko Mäkelä
befde6e97e MDEV-12353 preparation: Clean up page_cur_delete_rec()
page_cur_delete_rec(): Do not tolerate mtr=NULL.

page_delete_rec(): Merge with the only caller, RecIterator::remove().

RecIterator::m_mtr: New data member: a dummy mini-transaction.
2019-12-13 18:15:35 +02:00
Marko Mäkelä
1c282d4bc4 MDEV-16678: Ignore stray #sql-ib tables
The adjustments in adb117cf69
turned out to be insufficient.
2019-12-13 18:15:34 +02:00
Marko Mäkelä
8fa759a576 Merge 10.3 into 10.4
We disable the MDEV-21189 test galera.galera_partition
because it times out.
2019-12-13 17:30:37 +02:00
Anel Husakovic
8129ff1440 PR #1127 and PR #1150
PR#1127: Fix is_check_constraints.result to be compatibile with 10.3

The patch is done according to the original patch for MDEV-14474
1edd09c325 and not one which is merged on server
d526679efd.
This patch includes:
- Rename from `is_check_constraint` to `is_check_constraints` to tests
and results
- Per review, change the order of fields in IS check_constraints table by adding
the column `table_name` before `constraint_name`. According to the standard
2006 there is no `table_name` column.
- Original patch and one in `10.3` supports embedded server this patch doesn't
support. After the merge `10.3` will not support also.
- Don't use patch c8b8b01b61 to change the length of `CHECK_CLAUSE` field

PR#1150: MDEV-18440: Information_schema.check_constraints possible data leak

This patch is extension of PR 1127 and includes:
- Check for table grants
- Additional test according to the MDEV specification

Signed-off-by: Vicențiu Ciorbaru <vicentiu@mariadb.org>
2019-12-13 16:38:14 +02:00
Vicențiu Ciorbaru
193b5ed50b Merge branch '10.2' into 10.3 2019-12-13 16:36:46 +02:00
Eugene Kosov
a9ea0056c7 MDEV-21133: use aligned memcpy in redo log and buffer pool 2019-12-13 21:03:50 +07:00
Anel Husakovic
a134f1ebb1 PR #1127 and PR #1150
PR#1127: Fix is_check_constraints.result to be compatibile with 10.3

The patch is done according to the original patch for MDEV-14474
1edd09c325 and not one which is merged on server
d526679efd.
This patch includes:
- Rename from `is_check_constraint` to `is_check_constraints` to tests
and results
- Per review, change the order of fields in IS check_constraints table by adding
the column `table_name` before `constraint_name`. According to the standard
2006 there is no `table_name` column.
- Original patch and one in `10.3` supports embedded server this patch doesn't
support. After the merge `10.3` will not support also.
- Don't use patch c8b8b01b61 to change the length of `CHECK_CLAUSE` field

PR#1150: MDEV-18440: Information_schema.check_constraints possible data leak

This patch is extension of PR 1127 and includes:
- Check for table grants
- Additional test according to the MDEV specification
2019-12-13 15:58:30 +02:00
Sergei Golubchik
794911a27a tokudb: disable check_huge_pages_in_practice()
crashes on Debian 10
2019-12-13 11:24:23 +01:00
Sergei Golubchik
91c3d99804 tokudb: fix to compile with gcc 9.2.0 2019-12-13 11:24:23 +01:00
Alexander Barkov
bbd2fa5c65 MDEV-21278 Assertion is_unsigned() == attr.unsigned_flag' or Assertion field.is_sane()' failed
The type handler and unsigned_flag erroneously went out of sync in
Item_func_minus::fix_length_and_dec.
2019-12-13 12:45:16 +04:00
Marko Mäkelä
3466b47b0d Merge 10.2 into 10.3 2019-12-13 10:08:57 +02:00
Eugene Kosov
f0aa073f2b MDEV-20950 Reduce size of record offsets
offset_t: this is a type which represents one record offset.
It's unsigned short int.

a lot of functions: replace ulint with offset_t

btr_pcur_restore_position_func(),
page_validate(),
row_ins_scan_sec_index_for_duplicate(),
row_upd_clust_rec_by_insert_inherit_func(),
row_vers_impl_x_locked_low(),
trx_undo_prev_version_build():
  allocate record offsets on the stack instead of waiting for rec_get_offsets()
  to allocate it from mem_heap_t. So, reducing  memory allocations.

RECORD_OFFSET, INDEX_OFFSET:
  now it's less convenient to store pointers in offset_t*
  array. One pointer occupies now several offset_t. And those constant are start
  indexes into array to places where to store pointer values

REC_OFFS_HEADER_SIZE: adjusted for the new reality

REC_OFFS_NORMAL_SIZE:
  increase size from 100 to 300 which means less heap allocations.
  And sizeof(offset_t[REC_OFFS_NORMAL_SIZE]) now is 600 bytes which
  is smaller than previous 800 bytes.

REC_OFFS_SEC_INDEX_SIZE: adjusted for the new reality

rem0rec.h, rem0rec.ic, rem0rec.cc:
  various arguments, return values and local variables types were changed to
  fix numerous integer conversions issues.

enum field_type_t:
  offset types concept was introduces which replaces old offset flags stuff.
  Like in earlier version, 2 upper bits are used to store offset type.
  And this enum represents those types.

REC_OFFS_SQL_NULL, REC_OFFS_MASK: removed

get_type(), set_type(), get_value(), combine():
  these are convenience functions to work with offsets and it's types

rec_offs_base()[0]:
  still uses an old scheme with flags REC_OFFS_COMPACT and REC_OFFS_EXTERNAL

rec_offs_base()[i]:
  these have type offset_t now. Two upper bits contains type.
2019-12-13 00:26:50 +07:00
Eugene Kosov
014e125830 optimize crash recovery
recv_dblwr_t::list is used for appending to the beginning and iterating
through its elements. std::deque fits better for that purpose because
it does less allocations than std::forward_list and provides better memory
locality.
2019-12-12 22:19:41 +07:00
Eugene Kosov
ce41a9075a fix a memory leak introduced by f4b4284650 2019-12-12 21:29:51 +07:00
Marko Mäkelä
0a20e5ab77 Merge 10.2 into 10.3 2019-12-12 14:41:51 +02:00
Alexander Barkov
e0f9540bcc MDEV-20667 Server crash on pop_cursor
When backpatching a forward GOTO label, the old code erroneously
used CURSOR/HANDLER difference between context frames "c" and "a" to tune
a cpop/hpop command. So the cpop/hpop command later tried to pop
all cursors/handlers declared between "a" and "c", but those between
"b" and "c" were not cpushed/hpoped yet during the execution of "GOTO x".

Fixing the code to use the difference between frames "b" and "a" only.

BEGIN     -- a
 ...
GOTO x;   -- b
 ...
<<x>>     -- c
 ...
END       -- d
2019-12-12 16:25:16 +04:00
Vladislav Vaintroub
3304004a57 MDEV-21260 Innodb/Wjndows do not report error when trying open volumes on UNC paths
fil_node_t::find_metadata() tries to find out whether file
is on an SSD, and the disk sector size.
On Windows, it opens the corresponding volume for finding this data.

This does not go well, if datadir is on network path/UNC. The volume name
is invalid, CreateFile() function fails, and a cryptic (from the end user
perspective) error is reported. Like this

[ERROR] InnoDB: File \\.\\\workpc\work: 'CreateFile()' returned OS error 203.

The fix is not to report error if open volume failed, and the path was not
on fixed disk, i.e not on HDD or SSD. This is not a fatal error, there is
a fallback anyway.
2019-12-12 11:50:18 +01:00