Commit graph

191326 commits

Author SHA1 Message Date
Nikita Malyavin
d9e00770a3 MDEV-22608 ASAN use-after-poison in TABLE::check_period_overlaps
The bug was fixed by MDEV-22599 bugfix, which changed `Field::cmp` call
to `Field::cmp_prefix` in `TABLE::check_period_overlaps`.

The trick is that `Field_bit::cmp` apparently calls `Field_bit::cmp_key`,
which condiders an argument an actual pointer to data, which isn't correct
for `Field_bit`, since it stores data by `bit_ptr`. which is in the
beginning of the record, and using `ptr` is incorrect (we use it through
`ptr_in_record` call)
2020-11-02 14:11:43 +10:00
Nikita Malyavin
afca976885 MDEV-22639 Assertion failed in ha_check_overlaps upon multi-table update
After Sergei's cleanup this assertion is not actual anymore -- we can't
predict if the handler was used for lookup, especially in multi-update
scenario.

`position(old_data)` is made earlier in `ha_check_overlaps`, therefore it
is guaranteed that we compare right refs.
2020-11-02 14:11:43 +10:00
Nikita Malyavin
d543363f25 MDEV-22714 Assertion failed upon multi-update on table WITHOUT OVERLAPS
The problem here was that ha_check_overlaps internally uses ha_index_read,
which in case of fail overwrites table->status. Even though the handlers
are different, they share a common table, so the value is anyway spoiled.
This is bad, and table->status is badly designed and overweighted by
functionality, but nothing can be done with it, since the code related to
this logic is ancient and it's impossible to extract it with normal effort.

So let's just save and restore the value in ha_update_row before and after
the checks.

Other operations like INSERT and simple UPDATE are not in risk, since they
don't use this table->status approach.
DELETE does not do any unique checks, so it's also safe.
2020-11-02 14:11:42 +10:00
Nikita Malyavin
30894fe9a9 Add DBUG_ASSERT in Field::ptr_in_record
1. Subtracting table->record[0] from record is UB (non-contiguous buffers)
2. It is very popular to use move_field_offset, which changes Field::ptr,
but leaves table->record[0] unchanged. This makes a ptr_in_record result
incorrect, since it relies on table->record[0] value.
The check ensures the result is within the queried record boundaries.
2020-11-02 14:11:42 +10:00
Elena Stepanova
95fcd567bd List of unstable tests for 10.5.7 release 2020-11-02 04:15:13 +02:00
Elena Stepanova
1f18e0c70e List of unstable tests for 10.4.16 release 2020-11-02 01:29:52 +02:00
Oleksandr Byelkin
8e1e2856f2 Merge branch '10.4' into 10.5 2020-11-01 14:26:15 +01:00
Sergei Petrunia
a593e03d58 Add dbug_print_sel_arg() debugging help function 2020-11-01 14:33:57 +03:00
Elena Stepanova
d5ce782444 List of unstable tests for 10.3.26 release 2020-11-01 02:56:29 +02:00
Daniel Black
d6ea03fa94 MDEV-23630: mysqldump logically dump system table information
Add --system={all, users, plugins, udfs, servers, stats, timezones}

This will dump system information from the server in
a logical form like:
* CREATE USER
* GRANT
* SET DEFAULT ROLE
* CREATE ROLE
* CREATE SERVER
* INSTALL PLUGIN
* CREATE FUNCTION

"stats" is the innodb statistics tables or EITS and
these are dumped as INSERT/REPLACE INTO statements
without recreating the table.

"timezones" is the collection of timezone tables
which are important to transfer to generate identical
results on restoration.

Two other options have an effect on the SQL generated by
--system=all. These are mutually exclusive of each other.
* --replace
* --insert-ignore

--replace will include "OR REPLACE" into the logical form
like:
* CREATE OR REPLACE USER ...
* DROP ROLE IF EXISTS (MySQL-8.0+)
* CREATE OR REPLACE ROLE ...
* UNINSTALL PLUGIN IF EXISTS (10.4+) ... (before INSTALL PLUGIN)
* DROP FUNCTION IF EXISTS (MySQL-5.7+)
* CREATE OR REPLACE [AGGREGATE] FUNCTION
* CREATE OR REPLACE SERVER

--insert-ignore uses the construct " IF NOT EXISTS" where
supported in the logical syntax.

'CREATE OR REPLACE USER' includes protection against
being run as the same user that is importing the mysqldump.

Includes experimental support for dumping mysql-5.7/8.0
system tables and exporting logical SQL compatible with MySQL.

Updates mysqldump man page, including this information and
(removing obsolute bug reference)

Reviewed-by: anel@mariadb.org
2020-11-01 08:04:36 +11:00
Oleksandr Byelkin
80c951ce28 Merge branch '10.3' into 10.4 2020-10-31 21:06:49 +01:00
Elena Stepanova
6d3792a9a2 List of unstable tests for 10.2.35 release 2020-10-31 19:49:24 +02:00
Daniel Black
5b779c220d MDEV-22974: mysql_native_password make "invalid" valid
Per b9f3f06857, mysql_system_tables_data.sql creates
a mysql_native_password with a salted hash of "invalid" so that `set password`
will detect a native password can be applied:.

SHOW CREATE USER; diligently uses this value in its output
generating the SQL:

   MariaDB [(none)]> show create user;

   +---------------------------------------------------------------------------------------------------+
   | CREATE USER for dan@localhost                                                                     |
   +---------------------------------------------------------------------------------------------------+
   | CREATE USER `dan`@`localhost` IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket |
   +---------------------------------------------------------------------------------------------------+

Attempting to execute this before this patch results in:

  MariaDB [(none)]>  CREATE USER `dan2`@`localhost` IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket;
  ERROR 1372 (HY000): Password hash should be a 41-digit hexadecimal number

As such, deep the implementation of mysql_native_password we make "invalid" valid (pun intended)
such that the above create user will succeed. We do this by storing
"*THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE" (credit: Oracle MySQL), that is of an INCORRECT
length for a scramble.

In native_password_authenticate we check the length of this cached value
and immediately fail if it is anything other than the scramble length.

native_password_get_salt is only called in the context of set_user_salt, so all setting of native
passwords to hashed content of 'invalid', quite literally create an invalid password.

So other forms of "invalid" are valid SQL in creating invalid passwords:

   MariaDB [(none)]> set password = 'invalid';
   Query OK, 0 rows affected (0.001 sec)

   MariaDB [(none)]> alter user dan@localhost IDENTIFIED BY PASSWORD 'invalid';
   Query OK, 0 rows affected (0.000 sec)

closes #1628

Reviewer: serg@mariadb.com
2020-10-31 09:14:37 +11:00
Marko Mäkelä
b0ff791618 MDEV-24054 Assertion in_LRU_list failed in buf_flush_try_neighbors()
buf_flush_try_neighbors(): Before invoking buf_page_t::ready_for_flush(),
check that the freshly looked up buf_pool.page_hash entry actually is
a buffer page and not a buf_pool.watch[] sentinel for purge buffering.

This race condition was introduced in MDEV-15053
(commit b1ab211dee).
It is rather hard to hit this bug, because
buf_flush_check_neighbors() already checked the condition.
The problem exists if buf_pool.watch_set() was invoked for
a page in the range after the check in buf_flush_check_neighbor()
had been finished.
2020-10-30 19:06:50 +02:00
Oleksandr Byelkin
794f665139 Merge branch '10.2' into 10.3 2020-10-30 17:23:53 +01:00
Marko Mäkelä
03357ded17 Merge 10.4 into 10.5 2020-10-30 13:53:10 +02:00
Marko Mäkelä
1fddccf676 Update Connector/C 2020-10-30 13:47:56 +02:00
Marko Mäkelä
5b3be9e1c6 Try to stabilize main.innodb_ext_key,off
Thanks to Varun Gupta for suggesting this. This seems to
make main.innodb_ext_key,off more stable.
2020-10-30 13:47:56 +02:00
Marko Mäkelä
cb253b8687 MDEV-22387: Static_binary_string::q_append() invokes memcpy on NULL
Invoking memcpy() on a NULL pointer is undefined behaviour
(even if the length is 0) and gives the compiler permission to
assume that the pointer is nonnull. Recent versions of GCC
(starting with version 8) are more aggressively optimizing away
checks for NULL pointers. This undefined behaviour would cause
a SIGSEGV in the test main.func_encrypt on an optimized debug build
on GCC 10.2.0.
2020-10-30 13:07:42 +02:00
Marko Mäkelä
72eea39d4c MDEV-23991 fixup: Initialize the memory
This regression was introduced in
commit afc9d00c66.
This is a partial backport of
commit 199863d72b from 10.4.
2020-10-30 12:58:16 +02:00
Sergei Golubchik
066773e2f0 after-merge fix: update the test to pass in --ps 2020-10-30 11:46:12 +01:00
Marko Mäkelä
fbcd7c0c06 Update Connector/C 2020-10-30 12:22:23 +02:00
Varun Gupta
5a0c34e4c2 MDEV-24033: SIGSEGV in __memcmp_avx2_movbe from queue_insert | SIGSEGV in __memcmp_avx2_movbe from native_compare
The issue here was the system variable max_sort_length was being applied
to decimals and it was truncating the value for decimals to the number
of bytes set by max_sort_length.
This was leading to a buffer overflow as the values were written
to the buffer without truncation and then we moved the offset to
the number of bytes(set by max_sort_length), that are needed for comparison.

The fix is to not apply max_sort_length for fixed size types like INT,
DECIMALS and only apply max_sort_length for CHAR, VARCHARS, TEXT and
BLOBS.
2020-10-30 12:22:01 +02:00
Sergei Golubchik
c790218612 Fix RPM packaging on cmake 3.18+
cmake has caught up and since version 3.18 it started supporting
CPACK_RPM_POST_TRANS_SCRIPT_FILE, something we've supported for
two years and cmake 2.8.11. Both implementation add %posttrans tag
and rpmbuild gets confused.

Disable our implementation for cmake 3.18+
2020-10-30 10:32:59 +01:00
Marko Mäkelä
898521e2dd Merge 10.4 into 10.5 2020-10-30 11:15:30 +02:00
Marko Mäkelä
199863d72b MDEV-23991 fixup: Initialize the memory
Also, revert the work-around for the test that was attempted in
commit 85613a3247.

This issue was caught by MemorySanitizer as well as on the
Microsoft Windows debug builds, thanks to /MD being used
starting with 10.4.

The code fix will also be applied to 10.2 because the regression
was introduced in commit afc9d00c66.
2020-10-30 11:04:16 +02:00
Jan Lindström
5482d62760 Fix sporadic test failure on galera_parallel_apply_3nodes.
Test itself is not deterministic.
2020-10-30 09:19:29 +02:00
Jan Lindström
9936235985 MDEV-23659: Update Galera disabled.def file
Disable galera_var_replicate_myisam until fixed on 10.4
2020-10-30 08:54:05 +02:00
Jan Lindström
5485671474 Remove test that does not apply for 10.4. 2020-10-30 08:52:10 +02:00
Daniel Black
571bcf9aaa deb: logrotate - fix my_print_defaults arg
Corrects: 7803601dcb
2020-10-30 15:09:25 +11:00
Oleksandr Byelkin
a90b15837c MDEV-19838: fix of error messages 2020-10-29 22:20:21 +01:00
Oleksandr Byelkin
f9b0ee07ef MDEV-19838: followup, fix for PS & embedded
Use 9 byte (min length packet)
2020-10-29 22:19:32 +01:00
Monty
eb38e7ef60 MDEV-22879 SIGSEGV (or hang) in free/my_free
This bug was already fixed in a previous commit.
Added test case from the MDEV to prove it's fixed.
2020-10-29 19:20:10 +02:00
Monty
14d43f4fa6 MDEV-23222 SIGSEG in maria_create() because of double free
The crash happens because a double free in the case CREATE TABLE fails
because there is a conflicting tables on disk.

Fixed by ensuring that the double free can't happen.
2020-10-29 18:34:26 +02:00
Monty
4c99e3e948 Fixed bug in detection of getgrouplist parameters.
On my system, OpenSuse, I got a compilation error that some arguments
to getgrouplist() where not initialized
2020-10-29 17:36:49 +02:00
Marko Mäkelä
85613a3247 After-merge fix: main,innodb_ext_key,off
For some reason, in the test main,innodb_ext_key,off
we frequently get unexpected EXPLAIN output, in particular
on Microsoft Windows debug builders. Let us comment out that
EXPLAIN statement for now.
2020-10-29 16:27:04 +02:00
Monty
14798d3cd1 MDEV-23159 Assertion `table_share->tmp_table != NO_TMP_TABLE || m_lock_type != 2'...
The problem was that opt_sum_query() was, as part of MIN/MAX optimization,
doing read operations on constant tables that where already closed

Fixed by ensuring we don't try to read from tables that are closed.
2020-10-29 16:22:30 +02:00
Oleksandr Byelkin
2e5450af05 Merge branch '10.1' into 10.2 2020-10-29 15:16:53 +01:00
Marko Mäkelä
6d3356c12e MDEV-24053 MSAN use-of-uninitialized-value in tpool::simulated_aio::simulated_aio_callback()
Starting with commit ef3f71fa74
MemorySanitizer would complain that we are writing uninitialized
data via the doublewrite buffer.

buf_dblwr_t::add_to_batch(): Zero out any unused part of the
doublewrite buffer, for PAGE_COMPRESSED and ROW_FORMAT=COMPRESSED
tables.

Reviewed by: Eugene Kosov
2020-10-29 15:55:07 +02:00
Vicențiu Ciorbaru
8cfdddac71 MYSQL_JSON: Update test case to omit .so or .dll extension 2020-10-29 15:01:33 +02:00
Vicențiu Ciorbaru
8b2800d076 Fix decimals to 0 for MySQL JSON
This prevents the clash between NOT_FIXED_DEC differing between server
and plugins if MYSQL_SERVER is not defined during plugin compilation.
2020-10-29 15:01:33 +02:00
Vicențiu Ciorbaru
f3c5a92490 Add type_mysql_json.so to debian packages 2020-10-29 15:01:33 +02:00
Vicențiu Ciorbaru
a041b94032 Move vers_type_timestamp within the CC file
It's a virtual method and it can't be inlined anyway. This allows type
plugins (mysql_json in particular) to use Type_handler_blob and / or
subclass it, without needing to explicitly expose the
vers_type_timestamp object.
2020-10-29 15:01:33 +02:00
Vicențiu Ciorbaru
76fabe816f Expose utf8mb4_bin charset for plugins
Cleanup other linker errors
2020-10-29 15:01:33 +02:00
Vicențiu Ciorbaru
17ec6d6ce1 Skip MYSQL_JSON related tests if the plugin is not compiled 2020-10-29 15:01:33 +02:00
Marko Mäkelä
7b2bb67113 Merge 10.3 into 10.4 2020-10-29 13:38:38 +02:00
Aleksey Midenkov
27b762e23d MDEV-22805 SIGSEGV in check_fields on UPDATE
Additional case for PS protocol: UPDATE is converted to multi-update
in mysql_multi_update_prepare().
2020-10-29 13:47:50 +03:00
Sergei Golubchik
9a4398b048 update columnstore 2020-10-29 10:06:32 +01:00
Sergei Golubchik
05bd281697 SPIDER storage engine plugin -> Stable 2020-10-29 10:03:15 +01:00
Sergei Golubchik
17cf27f5b6 remove non-working debug assert
and restore the test modified in the same commit
(the non-replication related deadlock will be reported separately)
2020-10-29 09:35:39 +01:00