Commit graph

188233 commits

Author SHA1 Message Date
Daniel Black
1749a68968 mtr: remove --binary from patch args
This causes problems on FreeBSD which doesn't have a patch
that supports this.

Linux and Windows don't require it either.

Was added in c39877071a without
explaination.
2020-04-13 13:49:39 +03:00
Sergei Golubchik
e40ed0e881 fix incorrect merge 2020-04-12 18:07:40 +02:00
Sergey Vojtovich
06219c2ad4 MDEV-21599 - plugins.server_audit fails sporadically in buildbot
Fixed a couple of race conditions in the test case to ensure stable order
of events. Also removed all sleeps. Test execution time is down from 18s
to 0.15s.

On disconnect audit event is triggered after control is returned to
mysqltest client. Which means mysqltest may issue more commands
concurrently before disconnect is actually logged.

Similar problem happens with regular query execution: an event is
triggered after control is returner to the client. Which may end
up with unstable order of events in different connections.

Delayed insert rows are enqueued separately and can either be combined
into single event or go as separate events. Reduced number of inserted
rows to 1 to stabilize result.

Also backported 2b3f6ab from 10.5.
2020-04-10 19:35:53 +04:00
Varun Gupta
c1394ab6b5 MDEV-22191: Range access is not picked when index_merge_sort_union is turned off
When index_merge_sort_union is turned off only ror scans were considered for range
scans, which is wrong.
To fix the problem ensure both ror scans and non ror scans are considered for range
access
2020-04-08 23:47:03 +05:30
mkaruza
edc3899d97 MDEV-22051: Protocol::end_statement(): Assertion `0' failed on Galera node upon DDL attempt with conflicting lock
If FTWRL is issued, DDL statements should report error back to user before
TOI is started.
2020-04-08 16:42:18 +03:00
Julius Goryavsky
c8e0c524af MDEV-20676: systemd script not working
When trying to start mariadb via systemctl, WSREP failed
to start mysqld for wsrep recovery, because the binary
"galera-recovery" is neither searching the mysqld in the
same folder as the binary itself nor in the path variable
but instead expects the root to be /usr/local/mysql.

This fix changes the current directory to the desired
directory before starting mysqld.
2020-04-08 13:02:09 +02:00
Sergei Golubchik
d565895bbd fix rocksdb compression detection
cmake module name is BZip2, not BZIP2.
variable names are WITH_ROCKSDB_zstd and WITH_ROCKSDB_snappy.
it's SNAPPY_FOUND in older cmake (and both in newer cmake).
also, make it verbose.
2020-04-08 11:22:37 +02:00
Marko Mäkelä
476966b3fb MDEV-21535 Unnecessarily large ha_innobase::records_in_range() scans
The bug was already fixed in
commit 1242eb3d32
but the test innodb.innodb_wl6326 had not been enabled.
2020-04-08 09:59:00 +03:00
Vlad Lesin
5836191c8f MDEV-21168: Active XA transactions stop slave from working after backup
was restored.

Optionally rollback prepared XA's on "mariabackup --prepare".

The fix MUST NOT be ported on 10.5+, as MDEV-742 fix solves the issue for
slaves.
2020-04-07 15:05:38 +03:00
Rasmus Johansson
5720db2b43 MDEV-22176 Add JUnit support to MTR to generate XML test result
A new parameter has been added called xml-report, with which the
filename of the XML file is given to which the XML result is
written. There is also xml-package for adding a package value in
the XML output. Example usage:
./mysql-test-run.pl main.events_bugs innodb.count_distinct
main.explain_json innodb.file_format_defaults json.json_no_table
--suite=main,innodb,json --force --xml-report=build123456789.xml
--xml-package=simpletestrun
2020-04-07 09:27:51 +00:00
Otto Kekäläinen
b6eabce139 Travis-CI: Shorten deb build log to keep it under 4 MB
There is a 4 MB hard limit on Travis-CI and build output needs to be less
than that. Silencing the 'make install' step gets rid of a lot of
"Installing.." and "Missing.." and removing all mysql-test files will
make the dh_missing warnings much shorter.
2020-04-06 09:07:02 +03:00
Otto Kekäläinen
28604c349b Travis-CI: Add missing build dependency dh-exec
Backported from 30b44aaec7.
2020-04-06 09:06:51 +03:00
Otto Kekäläinen
811e4409ce Travis-CI: Shorten deb build log to keep it under 4 MB
There is a 4 MB hard limit on Travis-CI and build output needs to be less
than that. Silencing the 'make install' step gets rid of a lot of
"Installing.." and "Missing.." and removing all mysql-test files will
make the dh_missing warnings much shorter.
2020-04-05 23:55:45 +03:00
Otto Kekäläinen
396e83d777 Travis-CI: Add missing build dependency dh-exec
Backported from 30b44aaec7.
2020-04-05 18:56:16 +03:00
Alexander E. Patrakov
241ac3f487 MDEV-21140 Make galera_recovery.sh work with fs.protected_regular = 1 (#1417)
The log file is opened as root since commit bb7a70c, so there is no need
to chown it.
2020-04-05 18:40:27 +03:00
Igor Babaev
21b8743734 MDEV-21673 Calling stored procedure twice in the same session causes MariaDB to crash
This bug could happen only with a stored procedure containing queries with
more than one reference to a CTE that used local variables / parameters.

This bug was the result of an incomplete merge of the fix for the bug
MDEV-17154. The merge covered usage of parameter markers occurred in a CTE
that was referenced more than once, but missed coverage of local variables.
2020-04-04 09:24:22 -07:00
Aleksey Midenkov
aa7f2578fc MDEV-21471 ER_CRASHED_ON_USAGE upon UPDATE FOR PORTION on Aria table
Turn read cache off for periodic update.

Like 498a96a4 says:

Aria with row_format=fixed uses IO_CACHE of type READ_CACHE for
sequential read in update loop. When history row is inserted inside
this loop the cache misses it and fails with error.

This applicable to any additional row inserts on UPDATE. In this case
it was initiated by UPDATE FOR PORTION.

Related to MDEV-20441.
2020-04-03 23:55:48 +03:00
Aleksey Midenkov
198af54bb9 MDEV-20494 Fix wrongly ignored error status 2020-04-03 23:55:48 +03:00
Aleksey Midenkov
76063c2a13 MDEV-20494 ER_NOT_FORM_FILE or assertion upon adding partition to period table
- Fixed mysql_prepare_create_table() constraint duplicate checking;
- Refactored period constraint handling in mysql_prepare_alter_table():
  * No need to allocate new objects;
  * Keep old constraint name but exclude it from dup checking by automatic_name;
- Some minor memory leaks fixed;
- Some conceptual TODOs.
2020-04-03 23:55:48 +03:00
Daniel Black
cd88a606f5 Correct FreeBSD cpuset_t type 2020-04-03 15:30:33 +02:00
Hannu Hartikainen
64b70b09e6 my.cnf: mention that config files must be *.cnf
It took me a long time to debug why my configs were not being loaded,
and judging from online discussions I'm not the only one. Make the
comment in the default my.cnf a bit more helpful.

The !includedir directive is implemented in mysys/my_default.c.

- f_extensions[] is a list of file extensions. It includes .ini and .cnf
  on Windows, and only .cnf on all other platforms.
- search_default_file_with_ext() contains the !includedir directive. It
  filters files in the directory to those matching f_extensions[].

This file should only be applicable on Unix-like platforms, so only
files with the .cnf extension are read.

Closes #1485
2020-04-03 14:08:59 +02:00
Vladislav Vaintroub
fbef428645 MDEV-22137 correct documentation of tcp_keepalive_time system variable 2020-04-03 12:49:50 +02:00
Aleksey Midenkov
0932c5804d MDEV-20515 multi-update tries to position updated table by null reference
Cause

Join tmp table inserts null row because of OUTER JOIN, that's
expected. Since `multi_update::prepare2()` converted
`Item_temptable_rowid` into `Item_field` (28dbdf3)
`multi_update::send_data()` accesses join tmp record directly and
treats it as a normal row ignoring null status of ref field. NULL ref
field is then treated as normal in `multi_update::do_updates()` which
tries to position updated table by reference 0.

Note that reference 0 may be valid reference and the first row of
table can be wrongly updated (see multi_update.test).

Fix

Do not add row into multi-update tmp table in case of null ref
field. Join tmp table does not have null_row status at this time (as
well as `STATUS_NULL_ROW`) and cannot be skipped by these properties
(see first comment in multi_update::send_data()). But it has all null
fields (including the ref field).
2020-04-02 20:48:38 +03:00
Aleksey Midenkov
ba34f409ad MDEV-21688 Assertion or ER_WARN_DATA_OUT_OF_RANGE upon ALTER on previously versioned table
Earlier skip of history row. Cleanup of dead code for VTMD.
2020-04-02 20:48:38 +03:00
Aleksey Midenkov
44c6c7a923 MDEV-21342 Assertion in set_ok_status() upon spatial field error on system-versioned table
SQL_SELECT::check_quick() returns error status only
test_quick_select() returns -1. Fix error handling when lower frames
throw error, but it is ignored by test_quick_select(). Fix return
status for out-of-memory errors which are obviously must be processed
as error in upper frames.
2020-04-02 20:48:38 +03:00
Nikita Malyavin
9149017bb8
MDEV-17091 - Assertion failed after dropping versioning
Assertion `old_part_id == m_last_part' failed in ha_partition::update_row or `part_id == m_last_part' in ha_partition::delete_row upon UPDATE/DELETE after dropping versioning

PRIMARY KEY change hadn't been treated as partition reorganization in case of partitioning by KEY() (without parameters).

* set `*partition_changed= true` in the described case.
* since add/drop system versioning does not affect alter_info->key_list, it required separate attention
2020-04-02 22:37:36 +10:00
Sergei Petrunia
a219006636 MDEV-22014: Rowid Filtering is not displayed well in the optimizer trace
- Print the rowid filters that are available for use with each table.
- Make print_best_access_for_table() print which filter it has picked.
- Make best_access_path() print the filter for considered ref accesses.
2020-04-02 11:50:47 +03:00
Sergey Vojtovich
b40b3720cb MDEV-21348 - column compression memory leak
The Storage-Engine Independent Column Compression does not call
deflateEnd() when deflate() does not return Z_STREAM_END.

This for instance happens when the data is already (externally)
compressed and deflate() needs more space than the original data.

This patch is based on contribution by Martijn Broenland.
2020-04-02 00:57:00 +04:00
Sergey Vojtovich
05e4a87c8b A better fix for edd7e7c 2020-04-01 16:25:07 +04:00
Marko Mäkelä
31eaa2029f MDEV-19740: Have MYSQL_MAINTAINER_MODE only enable -Werror
Let us enable all GCC and clang warnings independently of the
MYSQL_MAINTAINER_MODE setting for both Debug and RelWithDebInfo
builds, and have MYSQL_MAINTAINER_MODE only enable -Werror.

The default setting of MYSQL_MAINTAINER_MODE=AUTO will continue
to apply the -Werror only to CMAKE_BUILD_TYPE=Debug. To build
a debug version without -Werror, MYSQL_MAINTAINER_MODE=OFF can
be used.
2020-04-01 15:16:11 +03:00
Oleksandr Byelkin
cb4da5da74 MDEV-20604: Duplicate key value is silently truncated to 64 characters in print_keydup_error
Added indication of truncated string for "s" and "M" formats
2020-04-01 11:34:32 +02:00
Marko Mäkelä
a1846b7a64 MDEV-22035 Memory leak in main.mysqltest
The test main.mysqltest could crash or hang with
cmake -DWITH_ASAN=ON builds. The reason appears to be
a memory leak, which was found out by manually invoking

echo --replace_regex a > file
ASAN_OPTIONS=log_path=/dev/shm/asan mysqltest ... < file

and then examining the /dev/shm/asan.* file.
2020-04-01 12:08:45 +03:00
Marko Mäkelä
f813131c7b Merge 5.5 into 10.1 2020-04-01 10:24:36 +03:00
Marko Mäkelä
bc862c4ebe Merge 10.1 into 10.2 2020-04-01 09:19:37 +03:00
Marko Mäkelä
b1742a5c95 MDEV-13626: Improve innodb.xa_recovery_debug
Improve the test that was imported and adapted for MariaDB in
commit fb217449dc.

row_undo_step(): Move the DEBUG_SYNC point from trx_rollback_for_mysql().
This DEBUG_SYNC point is executed after rolling back one row.

trx_rollback_for_mysql(): Clarify the comments that describe the scenario,
and remove the DEBUG_SYNC point.

If the statement "if (trx->has_logged_persistent())" and its body are
removed from trx_rollback_for_mysql(), then the test
innodb.xa_recovery_debug will fail because the transaction would still
exist in the XA PREPARE state. If we allow the XA COMMIT statement
to succeed in the test, we would observe an incorrect state of the
XA transaction where the table would contain row (1,NULL). Depending
on whether the XA transaction was committed, the table should either
be empty or contain the record (1,1). The intermediate state of
(1,NULL) should never be observed after completed recovery.
2020-04-01 09:13:01 +03:00
Marko Mäkelä
fb217449dc MDEV-13626: Import and adapt innodb.xa_recovery_debug
Adapt the test that was added in
mysql/mysql-server@6b65d9032c
but omitted in commit 2e814d4702.

Instead of triggering a log checkpoint, we will only trigger
a redo log flush before killing the server.
Note: the mtr.commit() call in trx_rollback_for_mysql()
will not actually make the undo log header page state change durable.
A call to log_write_up_to(mtr.commit_lsn(), true) would do that.

It is unclear what the originally reported bug scenario was.
As long as innobase_rollback_by_xid() will not return without
ensuring that the redo log has been durably written, we should be safe.
2020-03-31 15:10:54 +03:00
Daniele Sciascia
bdcecfa22c MDEV-22021: Galera database could get inconsistent with rollback to savepoint
When binlog is disabled, WSREP will not behave correctly when
SAVEPOINT ROLLBACK is executed and we will not rollback transaction.
2020-03-31 14:18:21 +03:00
mkaruza
2d16452a31 MDEV-22021: Galera database could get inconsistent with rollback to savepoint
When binlog is disabled, WSREP will not behave correctly when
SAVEPOINT ROLLBACK is executed since we don't register handlers for such case.
Fixed by registering WSREP handlerton for SAVEPOINT related commands.
2020-03-31 09:59:37 +03:00
Eugene Kosov
5001487632 MDEV-22074 UBSAN: applying zero offset to null pointer in hash.c
The fix: return fast when no work should be performed.
2020-03-31 00:44:01 +03:00
Eugene Kosov
edd7e7c85d MDEV-22069 UBSAN: runtime error: member access within null pointer of type 'MY_DIR_HANDLE' in mysys/my_lib.c
This is an error handling bug. When opendir() fails dirh is NULL and
we shouldn't try to free it.
2020-03-31 00:44:01 +03:00
Marko Mäkelä
e2f1f88fa6 Merge 10.3 into 10.4 2020-03-30 14:50:23 +03:00
Marko Mäkelä
b092d35f13 MDEV-20590 Introduce a file format constraint to ALTER TABLE
If a table is altered using the MDEV-11369/MDEV-15562/MDEV-13134
ALGORITHM=INSTANT, it can force the table to use a non-canonical
format:

* A hidden metadata record at the start of the clustered index
is used to store each column's DEFAULT value. This makes it possible
to add new columns that have default values without rebuilding the table.

* Starting with MDEV-15562 in MariaDB Server 10.4, a BLOB in the
hidden metadata record is used to store column mappings. This makes
it possible to drop or reorder columns without rebuilding the table.
This also makes it possible to add columns to any position or drop
columns from any position in the table without rebuilding the table.

If a column is dropped without rebuilding the table, old records
will contain garbage in that column's former position, and new records
will be written with NULL values, empty strings, or dummy values.

This is generally not a problem. However, there may be cases where
users may want to avoid putting a table into this format.
For example, users may want to ensure that future UPDATE operations
after an ADD COLUMN will be performed in-place, to reduce write
amplification. (Instantly added columns are essentially always
variable-length.) Users might also want to avoid bugs similar to
MDEV-19916, or they may want to be able to export tables to
older versions of the server.

We will introduce the option innodb_instant_alter_column_allowed,
with the following values:

* never (0): Do not allow instant add/drop/reorder,
to maintain format compatibility with MariaDB 10.x and MySQL 5.x.
If the table (or partition) is not in the canonical format, then
any ALTER TABLE (even one that does not involve instant column
operations) will force a table rebuild.

* add_last (1, default in 10.3): Store a hidden metadata record that
allows columns to be appended to the table instantly (MDEV-11369).
In 10.4 or later, if the table (or partition) is not in this format,
then any ALTER TABLE (even one that does not involve column changes)
will force a table rebuild.

Starting with 10.4:

* add_drop_reorder (2, default): Like 'add_last', but allow the
metadata record to store a column map, to support instant
add/drop/reorder of columns (MDEV-15562).
2020-03-30 12:41:59 +03:00
Thirunarayanan Balathandayuthapani
f8ec3ba01b MDEV-21832 FORCE all partition to rebuild if any one of the
partition does rebuild

- In ha_innobase::commit_inplace_alter_table() assumes that all partition
should do the same kind of alter operations. During DDL, if one partition
requires table rebuild and other partition doesn't need rebuild
then all partition should be forced to rebuild.
2020-03-30 12:41:59 +03:00
Marko Mäkelä
67f2782413 Fix GCC -Wstringop-truncation 2020-03-30 12:16:07 +03:00
Marko Mäkelä
1a9b6c4c7f Merge 10.2 into 10.3 2020-03-30 11:12:56 +03:00
Varun Gupta
b11ff3d495 MDEV-22019: Sig 11 in next_breadth_first_tab | max_sort_length setting + double GROUP BY leads to crash
No need to create a temp table for aggregation if we have encountered some error.
2020-03-30 08:03:54 +05:30
Sergey Vojtovich
9eae063e79 num_worker_threads my_atomic to Atomic_counter 2020-03-27 11:41:46 +04:00
Sergey Vojtovich
e91a3ea732 shutdown_group_count my_atomic to Atomic_counter 2020-03-27 02:17:40 +04:00
Sergey Vojtovich
ed8bf7c98f next_timeout_check my_atomic to std::atomic 2020-03-27 02:08:53 +04:00
Monty
0fb84216a3 Make mysql.innodb_mysql_lock deterministic
MDEV-7861 main.innodb_mysql_lock fails sporadically in buildbot
2020-03-26 19:39:51 +02:00