Commit graph

184629 commits

Author SHA1 Message Date
Sergei Golubchik
65e3d08538 MDEV-7209 mroonga storage engine fails to build on OpenBSD
rename files `version` -> `version_full`
2021-07-27 20:44:46 +02:00
Sergei Golubchik
aaac2477ca Merge remote-tracking branch 'merge/merge-pcre' into 10.2 2021-07-27 18:51:32 +02:00
Sergei Golubchik
4683ce82d2 8.45 2021-07-27 18:45:35 +02:00
Sergei Golubchik
a32373b65a Merge remote-tracking branch 'connect/10.2' into 10.2 2021-07-27 18:25:42 +02:00
Marko Mäkelä
afe00bb7cc MDEV-25998 fixup: Avoid a hang
btr_scrub_start_space(): Avoid an unnecessary tablespace lookup
and related acquisition of fil_system->mutex. In MariaDB Server 10.3
we would get deadlocks between that mutex and a crypt_data mutex.

The fix was developed by Thirunarayanan Balathandayuthapani.
2021-07-27 10:44:01 +03:00
Marko Mäkelä
e11cae71fe MDEV-25998 fixup: Assert that a mutex is being held 2021-07-27 09:26:24 +03:00
Marko Mäkelä
da094188f6 MDEV-24393 InnoDB disregards --skip-external-locking
On POSIX systems, InnoDB would unconditionally acquire advisory locks
on the files that it opens. On Linux, this would be observable by
a large number of entries in /proc/locks.

Other storage engines would only acquire advisory locks on files
based on the Boolean configuration parameter external_locking.

Let InnoDB do the same.

NOTE: The --skip-external-locking is activated by default. To have
InnoDB acquire advisory locks, --external-locking must be specified.

Reviewed by: Sergei Golubchik
2021-07-27 08:52:59 +03:00
Marko Mäkelä
cf1fc59856 MDEV-25594: Improve debug checks
trx_t::will_lock: Changed the type to bool.

trx_t::is_autocommit_non_locking(): Replaces
trx_is_autocommit_non_locking().

trx_is_ac_nl_ro(): Remove (replaced with equivalent assertion expressions).

assert_trx_nonlocking_or_in_list(): Remove.
Replaced with at least as strict checks in each place.

check_trx_state(): Moved to a static function; partially replaced with
individual debug assertions implementing equivalent or stricter checks.

This is a backport of commit 7b51d11cca
from 10.5.
2021-07-27 08:52:01 +03:00
Jan Lindström
0bd9f755b7 MDEV-26062 : InnoDB: WSREP: referenced FK check fail: Lock wait index PRIMARY table schema.child_table
Problem was that not all normal error codes where not handled
after wsrep_row_upd_check_foreign_constraints() call. Furhermore,
debug assertion did not contain all normal error cases. Changed
ib:: calls to WSREP_ calls to use wsrep instrumentation.
2021-07-26 15:55:58 +03:00
Thirunarayanan Balathandayuthapani
ce870b2a2a MDEV-25998 InnoDB removes the tablespace from default encrypt list early
Problem:
=========
As a part of MDEV-14398 patch, InnoDB added and removed
the tablespace from default encrypt list. But InnoDB removes
the tablespace from the default encrypt list too early due to
i) other encryption thread working on the tablespace
ii) When tablespace is being flushed at the end of
key rotation

InnoDB fails to decrypt/encrypt the tablespace since
the tablespace removed too early and it leads to
test case failure.

Solution:
=========
Avoid the removal of tablespace from default_encrypt_list
only when
1) Another active encryption thread working on tablespace
2) Eligible for tablespace key rotation
3) Tablespace is in flushing phase

Removed the workaround in encryption.innodb_encryption_filekeys test case.
2021-07-26 16:01:33 +05:30
Eugene Kosov
0711a53a33 fix clang build: check alignment the other way 2021-07-26 12:37:25 +03:00
Rucha Deodhar
af0b26f9b7 MDEV-23786: Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())'
failed for TokuDB engine CREATE TABLE

Analysis: Assertion failure happens because the database doesn't exist to
create the table but ha_tokudb::create() still returns false.
So error is not reported.
Fix: Store the error state and report the error.
2021-07-26 11:28:06 +05:30
Elena Stepanova
f29b3d6d82 Some tests can take very long time when run with valgrind
Set tests to non-valgrind:
  oqgraph.social
  encryption.innodb-page_encryption
  binlog_encryption.encrypted_master
  innodb.innodb-page_compression_lz4
  main.lock_multi_bug38499
  main.lock_multi_bug38691
2021-07-24 21:32:52 +03:00
Olivier Bertrand
fc98a6d54a Fix compile error for centos 74 Modified taboccur.cpp 2021-07-24 18:31:59 +02:00
Olivier Bertrand
5049887c8d Replace Mongo2.jar and Mongo3.jar with ne versions 2021-07-24 18:24:20 +02:00
Olivier Bertrand
0f18135ec8 - Make user variable prefix recognized by IsArgJson and IsJson
modified:   storage/connect/bsonudf.cpp
  modified:   storage/connect/jsonudf.cpp

- Stringify option is now a ; separated list of columns
  modified:   storage/connect/json.cpp
  modified:   storage/connect/json.h
  modified:   storage/connect/mongo.h
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabcmg.cpp
  modified:   storage/connect/tabcmg.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabjmg.h
  modified:   storage/connect/tabjson.cpp

- PrepareColist not a static function anymore (+ typo)
  modified:   storage/connect/taboccur.cpp

- JDVC: Recognize schema (database) from a wrapper server
  modified:   storage/connect/tabjdbc.cpp
2021-07-24 16:28:57 +02:00
Marko Mäkelä
173e562dc2 MDEV-26228 ASAN heap-use-after-free with ON UPDATE CASCADE
In commit 83d2e0841e (MDEV-24041)
we failed to notice that in addition to the bug with
DELETE and ON DELETE CASCADE, there is another bug with
UPDATE and ON UPDATE CASCADE.

row_ins_foreign_fill_virtual(): Use the correct memory heap
for everything that will be reachable from the cascade->update
that we return to the caller.

Note: It is correct to use the shorter-lived cascade->heap for
rec_get_offsets(), because that memory will be abandoned when
row_ins_foreign_fill_virtual() returns.
2021-07-23 17:20:57 +03:00
Julius Goryavsky
4c4237e63f MDEV-26080 fixup: fixed .result file for galera_roles test (one word must be enclosed in single quotes). 2021-07-23 08:31:32 +02:00
Igor Babaev
2820ad1c22 MDEV-26202 Unexpected failure with query using indirectly a recursive CTE twice
This bug was fixed by the patch for bug MDEV-26025.
Only a new test case is added here.
2021-07-22 17:08:08 -07:00
Marko Mäkelä
742b3a0d39 MDEV-26205 Merge new release of InnoDB 5.7.35 to 10.2 2021-07-22 18:07:37 +03:00
Sachin Agarwal
236f825ebf Bug INNODB_FT_TOTAL_CACHE_SIZE NOT CAPPED WHEN SET TO 32000000
Problem:
Server throws OOM error when we execute twitter load with SELECTs
for UPDATE + UPDATES, and SELECT queries on tables with full-text
index.
FTS cache->total_memory store count of total memory allocated to FTS
cache (for all fulltext indexes on a table).
For each word in fts cache, we store doc-id & word position in a
node->ilist.
we increment cache->total_memory with size of doc-id & word position
whereas we allocate ilist in chuck of 16, 32 ,64 bytes or 1.2 times
of last size.
When we wil insert huge amount of data into the FTS aux index tables
then collectively these small chucks for each token become huge
unaccounted memory allocated for FTS cache.

Fix:
Incremented cache->total_memory by size of chunk allocated to
node->ilist.

RB: 25286
Reviewed by : Rahul Agarkar <rahul.agarkar@oracle.com>

mysql/mysql-server@7ab5707f1c
2021-07-22 18:05:23 +03:00
Jakub Łopuszański
c4295b9be9 Bug ONLINE RESIZING BUFFER POOL CAN CRASH CONCURRENT BP LOOKUP
This patch changes it so that we do not free old BP `page_hash`, but rather modify it's parameters, during resize.

RB: 26084
Reviewed-by: Marcin Babij <marcin.babij@oracle.com>
Reviewed-by: Yasufumi Kinoshita <yasufumi.kinoshita@oracle.com>

mysql/mysql-server@ea3adc6a11
2021-07-22 18:05:23 +03:00
Marko Mäkelä
efae374efa MDEV-26203 CREATE INDEX may enforce incorrect maximum column length
ha_innobase::prepare_inplace_alter_table(): Unless the table is
being rebuilt, determine the maximum column length based on the
current ROW_FORMAT of the table. When TABLE_SHARE (and the .frm file)
contains no explicit ROW_FORMAT, InnoDB table creation or rebuild
will use innodb_default_row_format.

Based on mysql/mysql-server@3287d33acd
2021-07-22 17:55:05 +03:00
Marko Mäkelä
8ad6971a1a Update libmariadb 2021-07-22 17:54:49 +03:00
Marko Mäkelä
124dc0d85b MDEV-25361 fixup: Fix integer type mismatch
InnoDB tablespace identifiers and page numbers are 32-bit numbers.
Let us use a 32-bit type for them in innochecksum.

The changes in commit 1918bdf32c
broke the build on 32-bit Windows.

Thanks to Vicențiu Ciorbaru for an initial version of this fixup.
2021-07-22 17:53:43 +03:00
Anel Husakovic
b30f26e3fe Record tempfiles_encrypted test failure 2021-07-22 09:19:18 +02:00
Igor Babaev
4aeb2b1c6c MDEV-26189 Missing handling of unknown column in WHERE of recursive CTE
SQL processor failed to catch references to unknown columns and other
errors of the phase of semantic analysis in the specification of a
hanging recursive CTE. This happened because the function
With_clause::prepare_unreferenced_elements() failed to detect a CTE as
a hanging CTE if the CTE was recursive.
Fixing this problem in the code of the mentioned function opened another
problem: EXPLAIN started including the lines for the specifications of
hanging recursive CTEs in its output. This problem also was fixed in this
patch.

Approved by Dmitry Shulga <dmitry.shulga@mariadb.com>
2021-07-21 08:29:31 -07:00
Hollow Man
bd711d4f3d Typo fixes in item_strfunc.cc 2021-07-21 10:31:15 +03:00
Heinz Wiesinger
751ebe44fd Add feature summary at the end of cmake.
This gives a short overview over found/missing dependencies as well
as enabled/disabled features.

Initial author Heinz Wiesinger <heinz@m2mobi.com>
Additions by Vicențiu Ciorbaru <vicentiu@mariadb.org>
* Report all plugins enabled via MYSQL_ADD_PLUGIN
* Simplify code. Eliminate duplication by making use of WITH_xxx
  variable values to set feature "ON" / "OFF" state.

Reviewed by: wlad@mariadb.com (code details) serg@mariadb.com (the idea)
2021-07-21 10:22:56 +03:00
Vicențiu Ciorbaru
2b017367c7 Delete unused MYSQL_CHECK_{LZ4|LZO}_STATIC macros 2021-07-21 10:22:56 +03:00
Igor Babaev
4c387945f0 MDEV-25565 Crash on 2-nd execution of SP/PS for query calculating window functions
from view

A crash of the server happened when executing a stored procedure whose the
only query calculated window functions over a mergeable view specified
as a select from non-mergeable view. The crash could be reproduced if
the window specifications of the window functions were identical and both
contained PARTITION lists and ORDER BY lists. A crash also happened on
the second execution of the prepared statement created for such query.
If to use derived tables or CTE instead of views the problem still
manifests itself crashing the server.

When optimizing the window specifications of a window function the
server can substitute the partition lists and the order lists for
the corresponding lists from another window specification in the case
when the lists are identical. This substitution is not permanent and should
be rolled back before the second execution. It was not done and this
ultimately led to a crash when resolving the column names at the second
execution of SP/PS.
2021-07-20 11:26:33 -07:00
Igor Babaev
872422dcbb MDEV-26025 Server crashes while executing query with CTE in PS/SP
This bug appeared after the patch for bug MDEV-23886. Due to this bug
execution of queries with CTEs used the same CTE at least twice via
prepared statements or with stored procedures caused crashes of the server.
It happened because the select created for any of not the first usage of
a CTE erroneously was not included into all_selects_list.
This patch corrects the patch applied to fix the bug MDEV-26108.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2021-07-20 10:37:37 -07:00
Eugene Kosov
7da1cfb07a avoid searching std::map twice in innochecksum 2021-07-20 19:32:33 +03:00
Eugene Kosov
1918bdf32c MDEV-25361 innochecksum must not report errors for freed pages
Store and maintain xdes pages always. And doesn't verify checksums for
freed pages.

innochecksum can work only with the first space file of multiple ones.
Tell about it and abort in case of not the first file.
2021-07-20 19:32:33 +03:00
Jagdeep Sidhu
5f8651ac23 Fix switch case statement in trx_flush_log_if_needed_low()
In commit 2e814d4702 on MariaDB 10.2
the switch case statement in trx_flush_log_if_needed_low() regressed.

Since 10.2 this code was refactored to have switches in descending
order, so value of 3 for innodb_flush_log_at_trx_commit is behaving
the same as value of 2, that is no FSYNC is being enforced during
COMMIT phase. The switch should however not be empty and cases 2 and 3
should not have the identical contents.

As per documentation, setting innodb_flush_log_at_trx_commit to 3
should do FSYNC to disk if innodb_flush_log_at_trx_commit is set to 3.
This fixes the regression so that the switch statement again does
what users expect the setting should do.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2021-07-20 16:05:40 +03:00
Vladislav Vaintroub
1cfcf32cd0 fix libmariadb compilation, on GCC. 2021-07-20 11:34:19 +02:00
Sergei Golubchik
ce2a2bff0f MDEV-20787 Script dgcov.pl does not work
When building with `make` gcov files use full path names,
when building with `ninja` gcov files use paths relative to the source root

in gcov_one_file() the current directory is somewhere under CMakeFiles/,
so if a file exists in the specified location, this location
must've been a full path name.
2021-07-20 10:29:53 +02:00
Sergei Golubchik
6638cf2e9e MDEV-20787 Script dgcov.pl does not work
For every file.gcda file, gcov <7.x created file.cc.gcda.gcov.
While gcov 7.x and 8.x create file.cc.gcov
And sometimes otherfile.h.gcov or otherfile.ic.gcov, for included files.

(gcov 9.x+ creates .json.gz files, see MDEV-26102)

So, we use `gcov -l` that will create file.cc.gcda##file.cc.gcov,
file.cc.gcda##otherfile.h.gcov, etc. And we search and parse all
those file.cc.gcda*.gcov files.
2021-07-20 10:24:12 +02:00
Vladislav Vaintroub
0151590d8f Update libmariadb 2021-07-20 07:44:04 +02:00
Igor Babaev
f053349797 MDEV-26135 Assertion failure when executing PS with a hanging recursive CTE
The bug affected execution of queries with With clauses containing so-called
hanging recursive CTEs in PREPARE mode. A CTE is hanging if it's not used
in the query. Preparation of a prepared statement from a query with a
hanging CTE caused a leak in the server and execution of this prepared
statement led to an assert failure of the server built in the debug mode.
This happened because the units specifying recursive CTEs erroneously were
not cleaned up if those CTEs were hanging.
The patch enforces cleanup of hanging recursive CTEs in the same way as
other hanging CTEs.

Approved by dmitry.shulga@mariadb.com
2021-07-19 10:35:23 -07:00
Sergei Golubchik
b7886f55eb fix mysqld_safe --help
put defaults* options first (and together).
list --defaults-group-suffix too
2021-07-19 16:46:25 +02:00
Dmitry Shulga
bab989ab38 MDEV-26145: Incorrect metadata is sent on running query with union in PS mode
Test cases like the following one produce different result sets if it's run
with and without th option --ps-protocol.

CREATE TABLE t1(a INT);
--enable_metadata
(SELECT MAX(a) FROM t1) UNION (SELECT MAX(a) FROM t1);
--disable_metadata
DROP TABLE t1;

Result sets differ in metadata for the query
  (SELECT MAX(a) FROM t1) UNION (SELECT MAX(a) FROM t1);

The reason for different content of query metadata is that for queries
with union the items being created on JOIN preparing phase is placed into
item_list from SELECT_LEX_UNIT whereas for queries without union item_list
from SELECT_LEX is used instead.
2021-07-19 19:25:11 +07:00
Nikita Malyavin
c47e4aab62 MDEV-23597 Assertion `marked_for_read()' failed while evaluating DEFAULT
The columns that are part of DEFAULT expression were not read-marked
in statements like UPDATE...SET b=DEFAULT.

The problem is `F(DEFAULT)` expression depends of the left-hand side of an
assignment. However, setup_fields accepts only right-hand side value.
Neither Item::fix_fields does.

Suchwise, b=DEFAULT(b) works fine, because Item_default_field has
information on what field it is default of:
    if (thd->mark_used_columns != MARK_COLUMNS_NONE)
      def_field->default_value->expr->update_used_tables();

in Item_default_value::fix_fields().

It is not reasonable to pass a left-hand side to Item:fix_fields, because
the case is rare, so the rewrite
  b= F(DEFAULT)  ->  b= F(DEFAULT(b))

is made instead.

Both UPDATE and multi-UPDATE are affected, however any form of INSERT
is not: it marks all the fields in DEFAULT expressions for read in
TABLE::mark_default_fields_for_write().
2021-07-16 13:31:19 +03:00
Sergei Petrunia
6a89f346de MDEV-25858: Query results are incorrect when indexes are added
If test_if_skip_sort_order() decides to use an index to produce required
ordering,  it should disable "Range Checked for each record" optimization.

This is because Range-Checked-for-each-record may decide to use an index
(or an index_merge) which will not produce the required ordering.
2021-07-15 18:56:08 +03:00
Oli Sennhauser
da495b1b69 Typo fix in extrabackup.cc and innobackupex.cc
Thanks to @shinguz for helping with this.
This a backport commit from 10.7
2021-07-15 10:42:54 +03:00
Nayuta Yanagisawa
0f6e170c34 MDEV-25985 Spider handle ">=" as ">" in some cases
The function spider_db_append_key_where_internal() converts
HA_READ_AFTER_KEY to '>'. The conversion seems to be correct for
single-column indexes because HA_READ_AFTER_KEY means "read the
key after the provided value."

However, how about multi-column indexes? Assume that there is a
multi-column index on c1 and c2 and we search with the condition
'c1 >= 100 AND c2 > 200'. The key_range.flag corresponds to the
search condition could be HA_READ_AFTER_KEY. In such a case,
we could not simply convert HA_READ_AFTER_KEY to '>'.

The correct conversion is to convert HA_READ_AFTER_KEY to '>'
only for the last column in key_part_map and to convert
HA_READ_AFTER_KEY to '>=' for the other column.

The similar discussion also applies to the conversion from
key_range.flag to a sign of inequality.
2021-07-14 12:08:37 +00:00
Alexey Bychko
165a6dc97a MDEV-26119 RPM packages on RHEL-8 require the latest minor
changed rpm db query to output only version for libsepol and not release/buildnumber
2021-07-13 14:30:40 +07:00
Nikita Malyavin
191cae2d0d MDEV-18249 ASSERT_COLUMN_MARKED_FOR_READ failed in ANALYZE TABLE
The problem is the same as in MDEV-18166: columns in virtual field
expression are not marked for read, while the field itself does.

field->register_field_in_read_map() should be called for read-marking all
fields.

The test is reproduced only in 10.4+, however the fix is applicable to
10.2+.
2021-07-12 22:00:40 +03:00
Nikita Malyavin
f64a4f672a follow-up MDEV-18166: rename marking functions
Reformulate mark_columns_used_by_index* function family in a more laconic
way:

mark_columns_used_by_index -> mark_index_columns
mark_columns_used_by_index_for_read_no_reset -> mark_index_columns_for_read
mark_columns_used_by_index_no_reset -> mark_index_columns_no_reset
static mark_index_columns -> do_mark_index_columns
2021-07-12 22:00:40 +03:00
Nikita Malyavin
0f6a5b4390 [2/2] MDEV-18166 ASSERT_COLUMN_MARKED_FOR_READ failed on tables with vcols
Several different test cases were failing under the same reason: the
fields in a vcol expression were not marked during marking columns of a key
contatining virtual column for read.

Fix: make marking columns of a key for read a special case where
register_field_in_read_map() is done instead of plain bitmap_set_bit().

Some test cases are only reproducible in 10.4+, but the fix is applicable
to 10.2+
2021-07-12 22:00:39 +03:00