Commit graph

182985 commits

Author SHA1 Message Date
Varun Gupta
891609b571 MDEV-21318: Wrong results with window functions and implicit grouping
The issue here is for degenerate joins we should execute the window
function but it is not getting executed in all the cases.

To get the window function values window function needs to be executed
always. This currently does not happen in few cases
where the join would return 0 or 1 row like
  1) IMPOSSIBLE WHERE
  2) MIN/MAX optimization
  3) EMPTY CONST TABLE

The fix is to make sure that window functions get executed
and the temporary table is setup for the execution of window functions
2019-12-26 15:18:49 +05:30
Alexander Barkov
3dfe1ba3b8 MDEV-21388 Wrong result of DAYNAME()=xxx in combination with condition_pushdown_for_derived=on 2019-12-24 15:05:52 +04:00
Eugene Kosov
cbc170adf2 MDEV-21362 do something with -fno-builtin-memcmp for rem0cmp.cc
innodb.cmake: restrict -fno-builtin-memcmp for GCC versions older that 4.6
where optimization issue was solved.
2019-12-24 17:03:27 +08:00
Jan Lindström
17b1b8118a MDEV-21189 : Dropping partition with 'wsrep_OSU_method=RSU' and 'SESSION sql_log_bin = 0' cases the galera node to hang
Test cleanup. Best practice for using RSU, is to isolate the node
up-front, so this test did not reflect real world scenario
2019-12-23 12:57:22 +02:00
Thirunarayanan Balathandayuthapani
90ba87cb9e MDEV-19176 Reduce the memory usage during recovery
- post-push to fix the compilation issue
2019-12-23 15:56:57 +05:30
Thirunarayanan Balathandayuthapani
bba59abb03 MDEV-19176 Reduce the memory usage during recovery
- Moved the recv_sys->heap memory condition inside recv_parse_log_recs().
So that, InnoDB can mark the status as STORE_NO earlier.

- InnoDB uses one third of buffer pool chunk size for reading the redo
log records. In that case, we can avoid the scenario where buffer ran
out of memory issue during recovery.
2019-12-23 15:51:02 +05:30
Marko Mäkelä
a59a015a75 Plug memory leaks from numa_get_mems_allowed() 2019-12-23 07:47:16 +02:00
Marko Mäkelä
73985d8301 Merge 10.1 into 10.2 2019-12-23 07:14:51 +02:00
Eugene Kosov
496532b5c5 MDEV-20950: Fix 32-bit Windows build 2019-12-21 21:36:25 +02:00
Alexey Botchkov
aaaf6ceb8b MDE-21369 rpl.rpl_timezone fails with valgrind: Use of uninitialised value.
It's not safe to chenge the THD::thread_id.
So send the thread_id as an argument to the mysql_audit_external_lock.
2019-12-21 13:46:33 +04:00
Sergei Golubchik
aade6e53d3 fix a bad merge
in 10.1+ one should use

MY_CHECK_AND_SET_COMPILER_FLAG("-Wno-address-of-packed-member")

and it's already done in storage/tokudb/PerconaFT/CMakeLists.txt
2019-12-20 19:47:31 +01:00
Marko Mäkelä
8174e68895 MDEV-21371 Assertion failure in page_rec_get_next_low() during innodb_gis.rtree_compress
A debug assertion that was added in
commit ed0793e096
turns out to be too strict. In the test innodb_gis.rtree_compress,4k
the function is sometimes being invoked by purge for a
spatial index root page that is not a leaf page (PAGE_LEVEL is 1).
2019-12-20 19:21:37 +02:00
Eugene Kosov
f6b58d2916 MDEV-21239 ASAN use-after-poison in a server shutdown in innodb.innodb_buffer_pool_resize
If we manually poison memory region, ASAN requires that we manually unpoison
it too. We didn't do it and munmap() didn't do it too. That was the issue.

os_mem_free_large(): unpoison memory region. Just in case.

https://github.com/google/sanitizers/issues/182
86acaa9457/compiler-rt/include/sanitizer/asan_interface.h (L21)
2019-12-20 19:59:58 +08:00
Alexander Barkov
1f1e3ce8a1 MDEV-21319 COUNT(*) returns 1, actual SELECT returns no result in 10.3.21, but 1 result in 10.1.41
Item_ref::val_(datetime|time)_packed() erroneously called
(*ref)->val_(datetime|time)_packed().

- Fixing to call (*ref)->val_(datetime|time)_packed_result().
- Backporting Item::val_(datetime|time)_packed_result() from 10.3.
- Fixing Item_field::get_date_result() to handle null_value in the same
  way how Item_field::get_date() does.
2019-12-19 14:03:54 +04:00
Eugene Kosov
a5a433e256 fix windows compilation 2019-12-18 23:35:33 +08:00
Eugene Kosov
3d3d8f1012 MDEV-21337 fix aligned_malloc()
do not fallback to malloc(), always return properly aligned buffer
2019-12-18 20:09:52 +08:00
Sergei Petrunia
984b3c1544 MDEV-21341: Fix UBSAN failures, part #3
One may not call memcpy(dst, src=NULL, size), even if size==0.
2019-12-18 13:11:07 +03:00
Sergei Petrunia
8b9db11718 MDEV-21341: Fix optimizer-related UBSAN failures, part #2
Remove Query_tables_list::lock_tables_state - it is not used and it causes
errors like this:

sql_lex.h:1675:7: runtime error: load of value 2779096485, which is not a
valid value for type 'enum_lock_tables_state'
2019-12-18 12:56:54 +03:00
Sergei Petrunia
0ac0bc8cb6 MDEV-21341: Fix optimizer-related UBSAN failures, part #1:
Fix wrong typecast
2019-12-18 12:34:14 +03:00
Jan Lindström
c3824766c5 Fortify galera_partition test. 2019-12-18 10:02:57 +02:00
Jan Lindström
e9259d50f0 MDEV-21335 : Galera test failure on suite wsrep
Problem was that wsrep_on was OFF.
2019-12-18 10:02:57 +02:00
Vladislav Vaintroub
189fa30085 MDEV-21343 Threadpool/Unix- wait_begin() function does not wake/create threads, when it should
Fixed the condition for waking up/creating another thread.

If there is some work to do (if the request queue is not empty),
a thread should be woken or created.

The condition was incorrect since 18c9b345b4
2019-12-17 21:57:40 +01:00
Sergei Golubchik
0d70f40bdb Disable -Werror for rocksdb submodule
We cannot have -Werror for third-party submodules because
whenever they break the build we cannot fix it as needed.
2019-12-17 21:22:57 +01:00
Varun Gupta
164cf4f463 MDEV-16579: Wrong result of query using DISTINCT COUNT(*) OVER (*)
The query requires 2 temporary tables for execution, the window function
is always attached to the last temporary table, but in this case the
result field of the window function points to the first temporary table
rather than the last one.
Fixed this by not changing window function items with temporary table
items of the first temporary table.
2019-12-17 21:45:53 +05:30
Alexander Barkov
3d98892232 Merge remote-tracking branch 'origin/5.5' into 10.1 2019-12-16 13:08:17 +04:00
Alexander Barkov
fc860d3fa3 MDEV-21065 UNIQUE constraint causes a query with string comparison to omit a row in the result set 2019-12-16 12:57:08 +04:00
Anel Husakovic
8129ff1440 PR #1127 and PR #1150
PR#1127: Fix is_check_constraints.result to be compatibile with 10.3

The patch is done according to the original patch for MDEV-14474
1edd09c325 and not one which is merged on server
d526679efd.
This patch includes:
- Rename from `is_check_constraint` to `is_check_constraints` to tests
and results
- Per review, change the order of fields in IS check_constraints table by adding
the column `table_name` before `constraint_name`. According to the standard
2006 there is no `table_name` column.
- Original patch and one in `10.3` supports embedded server this patch doesn't
support. After the merge `10.3` will not support also.
- Don't use patch c8b8b01b61 to change the length of `CHECK_CLAUSE` field

PR#1150: MDEV-18440: Information_schema.check_constraints possible data leak

This patch is extension of PR 1127 and includes:
- Check for table grants
- Additional test according to the MDEV specification

Signed-off-by: Vicențiu Ciorbaru <vicentiu@mariadb.org>
2019-12-13 16:38:14 +02:00
Sergei Golubchik
794911a27a tokudb: disable check_huge_pages_in_practice()
crashes on Debian 10
2019-12-13 11:24:23 +01:00
Sergei Golubchik
91c3d99804 tokudb: fix to compile with gcc 9.2.0 2019-12-13 11:24:23 +01:00
Eugene Kosov
f0aa073f2b MDEV-20950 Reduce size of record offsets
offset_t: this is a type which represents one record offset.
It's unsigned short int.

a lot of functions: replace ulint with offset_t

btr_pcur_restore_position_func(),
page_validate(),
row_ins_scan_sec_index_for_duplicate(),
row_upd_clust_rec_by_insert_inherit_func(),
row_vers_impl_x_locked_low(),
trx_undo_prev_version_build():
  allocate record offsets on the stack instead of waiting for rec_get_offsets()
  to allocate it from mem_heap_t. So, reducing  memory allocations.

RECORD_OFFSET, INDEX_OFFSET:
  now it's less convenient to store pointers in offset_t*
  array. One pointer occupies now several offset_t. And those constant are start
  indexes into array to places where to store pointer values

REC_OFFS_HEADER_SIZE: adjusted for the new reality

REC_OFFS_NORMAL_SIZE:
  increase size from 100 to 300 which means less heap allocations.
  And sizeof(offset_t[REC_OFFS_NORMAL_SIZE]) now is 600 bytes which
  is smaller than previous 800 bytes.

REC_OFFS_SEC_INDEX_SIZE: adjusted for the new reality

rem0rec.h, rem0rec.ic, rem0rec.cc:
  various arguments, return values and local variables types were changed to
  fix numerous integer conversions issues.

enum field_type_t:
  offset types concept was introduces which replaces old offset flags stuff.
  Like in earlier version, 2 upper bits are used to store offset type.
  And this enum represents those types.

REC_OFFS_SQL_NULL, REC_OFFS_MASK: removed

get_type(), set_type(), get_value(), combine():
  these are convenience functions to work with offsets and it's types

rec_offs_base()[0]:
  still uses an old scheme with flags REC_OFFS_COMPACT and REC_OFFS_EXTERNAL

rec_offs_base()[i]:
  these have type offset_t now. Two upper bits contains type.
2019-12-13 00:26:50 +07:00
Vlad Lesin
beec9c0e19 MDEV-21255: Deadlock of parallel slave and mariabackup (with failed log
copy thread)

mariabackup hangs waiting until innodb redo log thread read log till certain
LSN, and it waits under FTWRL. If there is redo log read error in the thread,
it is finished, and main thread knows nothing about it, what leads to hanging.
As it hangs under FTWRL, slave threads on server side can be blocked due
to MDL lock conflict.

The fix is to finish mariabackup with error message on innodb redo log read
failure.
2019-12-12 13:28:30 +03:00
Vladislav Vaintroub
a3a8360d57 CMake,Windows - cleanup data directory prior to bootstrap for nitial_database target 2019-12-12 00:05:55 +01:00
Varun Gupta
808036a61d MDEV-19380: ASAN heap-use-after-free in Protocol::net_store_data
The issue here is window function makes the passed string object
to point to an area in a temporary table's record buffer.
Then, the temporary table is freed, together with its record buffer.
Then, Item_cache_str attempts to read this value.

The fix is to call value_buff.copy(). This will make the value_buff to store
its string in a buffer that it owns, which will not disappear unexpectedly.
2019-12-12 03:55:46 +05:30
Daniel Bartholomew
546644f1cc
bump the VERSION 2019-12-11 13:00:49 -05:00
Vladislav Vaintroub
8d2f6d3ca5 Fix overly chatty connect cmake, once again 2019-12-11 16:57:59 +01:00
Vladislav Vaintroub
202a62deb0 MDEV-11345 Compile english error messages into mysqld executable.
Simplify loading messages into mariabackup. Do the same as server does
We're forcing english, so there is no attempt to load errmsg.sys
2019-12-11 16:15:40 +01:00
Vladislav Vaintroub
280f1c2605 MDEV-11345 Compile english error messages into mysqld executable. 2019-12-11 11:13:32 +01:00
Marko Mäkelä
f2d3b2eede Cleanup test sys_vars.innodb_buffer_pool_size_basic
When using huge pages, the innodb_buffer_pool_size cannot necessarily
be restored. Simplify things by restarting the server.
2019-12-10 17:01:36 +02:00
Marko Mäkelä
41e6a154ec MDEV-14482 - Cache line contention on ut_rnd_interval()
InnoDB RNG maintains global state, causing otherwise unnecessary bus
traffic. Even worse, this is cross-mutex traffic. That is, different
mutexes suffer from contention.

Fixed delay of 4 was verified to give best throughput by OLTP update
index and read-write benchmarks on Intel Broadwell (2/20/40) and
ARM (1/46/46).

This is a backport of ce04790065 from
MariaDB Server 10.3.
2019-12-10 17:01:36 +02:00
Marko Mäkelä
b1f2d3a8c8 MDEV-21256: Replace the 64-bit LCG with a 32-bit Galois LFSR
We should not need anywhere near 32 bits of entropy, so we might
just limit ourselves to a 32-bit random number generator.

Also, it might be cheaper to use exclusive-or, bit shifting and
conditional jumps, instead of multiplication and addition.

We use relaxed atomic operations on the global random number generator
state in order in an attempt to silence any warnings about race conditions.
There is an obvious race condition between the load and store in
ut_rnd_gen(), but we do not think that it matters much that the
state of the random number generator could 'stutter'.

This change seems makes the 'uncompress_ops' nondeterministic
in innodb_zip.cmp_per_index after the restart. It looks like
there is an inherent race condition in the test, because the
table could be opened for InnoDB statistics recalculation
already before innodb_cmp_per_index_enabled was set. We might
end up having uncompress_ops anywhere between 0 and 9, or perhaps
even more. Let us remove that part of the test.
2019-12-10 16:59:34 +02:00
Marko Mäkelä
d146e3dcfe MDEV-21256: Simplify ut_rnd_interval()
ut_rnd_interval(): Remove the first parameter, which was mostly
passed as 0. Implement as a simple wrapper around ut_rnd_gen().
Trivially return 0 if the size of the interval is smaller than 2.

ut_rnd_ulint_counter, ut_rnd_gen_next_ulint(), ut_rnd_gen_ulint(): Remove.
2019-12-10 16:58:28 +02:00
Marko Mäkelä
51fc8ab73e MDEV-21256: Reduce the use of ut_rnd_gen_next_ulint()
ut_rnd_set_seed(): Unused function; remove.

ut_rnd_gen(): Renamed from page_cur_lcg_prng().

ut_rnd_current: The internal state of ut_rnd_gen().

page_cur_open_on_rnd_user_rec(): Replace linear search with
page_rec_get_nth().
2019-12-10 16:58:28 +02:00
Eugene Kosov
4c0854f221 MDEV-21223 innodb_fts.sync_ddl fails in buildbot, server crashed in que_thr_step
FreeState(): replace pointer to freed memory with NULL. This actually fixes a crash
which is use-after-free as reported by ASAN

DbugParse(): unconditionally lock mutex because we're touching shared init_settings.keywords
2019-12-10 21:16:21 +07:00
Oleksandr Byelkin
af650c76a6 MDEV-18460: Server crashed in strmake / tdc_create_key / THD::create_tmp_table_def_key
When there is a WITH clause we postpone check for tables without
database for later stages when tables in WITH will be defined.
But we should not try to open such tables as temporary tables because
temporary tables always belong to a some database.
2019-12-10 07:37:18 +01:00
Marko Mäkelä
425748f1b5 Cleanup: Replace a redundant statement with an assertion 2019-12-09 14:54:27 +02:00
Marko Mäkelä
292015d486 MDEV-21254 Remove unused keywords from the InnoDB SQL parser
The InnoDB internal SQL parser, which is used for updating the InnoDB
data dictionary tables (to be removed in MDEV-11655), persistent
statistics (to be refactored in MDEV-15020) and fulltext indexes,
implements some unused keywords and built-in functions:

OUT BINARY BLOB INTEGER FLOAT SUM DISTINCT READ
COMPACT BLOCK_SIZE
TO_CHAR TO_NUMBER BINARY_TO_NUMBER REPLSTR SYSDATE PRINTF ASSERT
RND RND_STR ROW_PRINTF UNSIGNED

Also, procedures are never declared with parameters. Only one top-level
procedure is declared and invoked at a time, and parameters are being
passed via pars_info_t.
2019-12-09 12:32:04 +02:00
Jan Lindström
59e14b9684 MDEV-21189: Dropping partition with 'wsrep_OSU_method=RSU' and 'SESSION sql_log_bin = 0' cases the galera node to hang
Found two bugs

(1) have_committing_connections was missing mutex unlock on one
exit case. As this function is called on a loop it caused mutex
lock when we already owned the mutex. This could cause hang.

(2) wsrep_RSU_begin did set up error code when partition to
be dropped could not be MDL-locked because of concurrent
operations but wrong error code was propagated to upper layer
causing error to be ignored. This could have also caused
the hang.
2019-12-09 08:14:39 +02:00
Alexey Botchkov
fd1979bc9a MDEV-18463 Don't allow multiple table CONSTRAINTs with the same name.
Add necessary checks.
2019-12-09 01:17:16 +04:00
Vladislav Vaintroub
6aa0fa3897 CONC-417 Windows clients using Schannel often encounter error SEC_E_INVALID_TOKEN
reenable ssl_8k_key on Windows, which was affected by this bug
2019-12-08 18:28:21 +01:00
Vladislav Vaintroub
6484288cd2 CONC-447 ERROR 2026 (HY000): SSL connection error: Certificate signature check failed
Enable CRL checking on Windows.
Enable certificate verification testing in client testing.
2019-12-08 18:26:16 +01:00