Commit graph

184742 commits

Author SHA1 Message Date
Sergei Golubchik
6638cf2e9e MDEV-20787 Script dgcov.pl does not work
For every file.gcda file, gcov <7.x created file.cc.gcda.gcov.
While gcov 7.x and 8.x create file.cc.gcov
And sometimes otherfile.h.gcov or otherfile.ic.gcov, for included files.

(gcov 9.x+ creates .json.gz files, see MDEV-26102)

So, we use `gcov -l` that will create file.cc.gcda##file.cc.gcov,
file.cc.gcda##otherfile.h.gcov, etc. And we search and parse all
those file.cc.gcda*.gcov files.
2021-07-20 10:24:12 +02:00
Vladislav Vaintroub
0151590d8f Update libmariadb 2021-07-20 07:44:04 +02:00
Igor Babaev
f053349797 MDEV-26135 Assertion failure when executing PS with a hanging recursive CTE
The bug affected execution of queries with With clauses containing so-called
hanging recursive CTEs in PREPARE mode. A CTE is hanging if it's not used
in the query. Preparation of a prepared statement from a query with a
hanging CTE caused a leak in the server and execution of this prepared
statement led to an assert failure of the server built in the debug mode.
This happened because the units specifying recursive CTEs erroneously were
not cleaned up if those CTEs were hanging.
The patch enforces cleanup of hanging recursive CTEs in the same way as
other hanging CTEs.

Approved by dmitry.shulga@mariadb.com
2021-07-19 10:35:23 -07:00
Sergei Golubchik
b7886f55eb fix mysqld_safe --help
put defaults* options first (and together).
list --defaults-group-suffix too
2021-07-19 16:46:25 +02:00
Dmitry Shulga
bab989ab38 MDEV-26145: Incorrect metadata is sent on running query with union in PS mode
Test cases like the following one produce different result sets if it's run
with and without th option --ps-protocol.

CREATE TABLE t1(a INT);
--enable_metadata
(SELECT MAX(a) FROM t1) UNION (SELECT MAX(a) FROM t1);
--disable_metadata
DROP TABLE t1;

Result sets differ in metadata for the query
  (SELECT MAX(a) FROM t1) UNION (SELECT MAX(a) FROM t1);

The reason for different content of query metadata is that for queries
with union the items being created on JOIN preparing phase is placed into
item_list from SELECT_LEX_UNIT whereas for queries without union item_list
from SELECT_LEX is used instead.
2021-07-19 19:25:11 +07:00
Nikita Malyavin
c47e4aab62 MDEV-23597 Assertion `marked_for_read()' failed while evaluating DEFAULT
The columns that are part of DEFAULT expression were not read-marked
in statements like UPDATE...SET b=DEFAULT.

The problem is `F(DEFAULT)` expression depends of the left-hand side of an
assignment. However, setup_fields accepts only right-hand side value.
Neither Item::fix_fields does.

Suchwise, b=DEFAULT(b) works fine, because Item_default_field has
information on what field it is default of:
    if (thd->mark_used_columns != MARK_COLUMNS_NONE)
      def_field->default_value->expr->update_used_tables();

in Item_default_value::fix_fields().

It is not reasonable to pass a left-hand side to Item:fix_fields, because
the case is rare, so the rewrite
  b= F(DEFAULT)  ->  b= F(DEFAULT(b))

is made instead.

Both UPDATE and multi-UPDATE are affected, however any form of INSERT
is not: it marks all the fields in DEFAULT expressions for read in
TABLE::mark_default_fields_for_write().
2021-07-16 13:31:19 +03:00
Sergei Petrunia
6a89f346de MDEV-25858: Query results are incorrect when indexes are added
If test_if_skip_sort_order() decides to use an index to produce required
ordering,  it should disable "Range Checked for each record" optimization.

This is because Range-Checked-for-each-record may decide to use an index
(or an index_merge) which will not produce the required ordering.
2021-07-15 18:56:08 +03:00
Oli Sennhauser
da495b1b69 Typo fix in extrabackup.cc and innobackupex.cc
Thanks to @shinguz for helping with this.
This a backport commit from 10.7
2021-07-15 10:42:54 +03:00
Nayuta Yanagisawa
0f6e170c34 MDEV-25985 Spider handle ">=" as ">" in some cases
The function spider_db_append_key_where_internal() converts
HA_READ_AFTER_KEY to '>'. The conversion seems to be correct for
single-column indexes because HA_READ_AFTER_KEY means "read the
key after the provided value."

However, how about multi-column indexes? Assume that there is a
multi-column index on c1 and c2 and we search with the condition
'c1 >= 100 AND c2 > 200'. The key_range.flag corresponds to the
search condition could be HA_READ_AFTER_KEY. In such a case,
we could not simply convert HA_READ_AFTER_KEY to '>'.

The correct conversion is to convert HA_READ_AFTER_KEY to '>'
only for the last column in key_part_map and to convert
HA_READ_AFTER_KEY to '>=' for the other column.

The similar discussion also applies to the conversion from
key_range.flag to a sign of inequality.
2021-07-14 12:08:37 +00:00
Alexey Bychko
165a6dc97a MDEV-26119 RPM packages on RHEL-8 require the latest minor
changed rpm db query to output only version for libsepol and not release/buildnumber
2021-07-13 14:30:40 +07:00
Nikita Malyavin
191cae2d0d MDEV-18249 ASSERT_COLUMN_MARKED_FOR_READ failed in ANALYZE TABLE
The problem is the same as in MDEV-18166: columns in virtual field
expression are not marked for read, while the field itself does.

field->register_field_in_read_map() should be called for read-marking all
fields.

The test is reproduced only in 10.4+, however the fix is applicable to
10.2+.
2021-07-12 22:00:40 +03:00
Nikita Malyavin
f64a4f672a follow-up MDEV-18166: rename marking functions
Reformulate mark_columns_used_by_index* function family in a more laconic
way:

mark_columns_used_by_index -> mark_index_columns
mark_columns_used_by_index_for_read_no_reset -> mark_index_columns_for_read
mark_columns_used_by_index_no_reset -> mark_index_columns_no_reset
static mark_index_columns -> do_mark_index_columns
2021-07-12 22:00:40 +03:00
Nikita Malyavin
0f6a5b4390 [2/2] MDEV-18166 ASSERT_COLUMN_MARKED_FOR_READ failed on tables with vcols
Several different test cases were failing under the same reason: the
fields in a vcol expression were not marked during marking columns of a key
contatining virtual column for read.

Fix: make marking columns of a key for read a special case where
register_field_in_read_map() is done instead of plain bitmap_set_bit().

Some test cases are only reproducible in 10.4+, but the fix is applicable
to 10.2+
2021-07-12 22:00:39 +03:00
Nikita Malyavin
7d9ba57da4 [1/2] MDEV-18166 ASSERT_COLUMN_MARKED_FOR_READ failed on tables with vcols
This is a 10.2+ part of a jira task

The two bugs regarding virtual column marking have been fixed:

1. UPDATE of a partitioned table, where the optimizer has chosen a
 secondary index to make a filesort;
2. INSERT into a table with a nonblob field generated from a blob, with
 binlog enabled and binlog_row_image=noblob.

3. DELETE from a view on a table with virtual column.

Generally the assertion happens from update_virtual_fields() call

These bugs are root-caused by missing field marking for dependant fields
of a virtual column.

Therefore a fix is: mark all the fields involved in the vcol expression by
calling field->register_field_in_read_map() instead just setting a single
bit.

3 was reproducible only on 10.4+, however the problem might has just been
invisible in the earlier versions. The fix is applicable to 10.2-10.3 as
well.
2021-07-12 22:00:39 +03:00
Nikita Malyavin
0e9ba176bf MDEV-17890 Server crash on DELETE with YEAR field with truncated expr
The failing reason was inconsistent truncation rules: the value of virtual
column could have been evaluated to '2000' sometimes instead of '0000' for
value 'a'.

The reason why `c YEAR AS ('aaaa')` was not evaluated same is that len=4 is
a special case insidew Field_year::store.

The correct fix is: always evaluate a bad value to 0000 instead 2000.
The truncated values should be evaluated as usual.

$support_virtual_index is finally changed to 1 in gcol.gcol_ins_upd_innodb,
which is also enough for testing.

The test from original bug report is also added.
2021-07-12 22:00:39 +03:00
Robert Bindar
b082716892 MDEV-21206 Can't link zlib library during DBD::mysql installation
mysql_config should never return -lzlib as linking flag.
Clients trying to link against mariadb that use the output of
mysql_config experience linking error because we return -lzlib
when we don't ship any libzlib.a in our binary tarballs, bundled
zlib static library is already linked.
2021-07-12 12:42:30 +03:00
Thirunarayanan Balathandayuthapani
fb0b28932c MDEV-25998 encryption.innodb_encryption_filekeys failure in buildbot
- Set the innodb_encrypt_tables variable before
timeout happens. It will add the pending tablespace
to default encrypt list and does the encryption/decryption
based on innodb_encrypt_tables.
2021-07-09 15:47:05 +05:30
Anel Husakovic
85063aebda Update description of mariadb-common package
- Commit https://github.com/mariadb/server/commit/438ed0408c69
introduced `mariadb-common` package and added description. Example in
description is confusing since files referred to are not installed and
not related/owned by the `mariadb-common` package.
- Patch is updating real file/directory description section pointing to
the real examples owned by the package.
- Example:
```
mariadb-client-10.3				install
mariadb-client-core-10.3			install
mariadb-common					install
mariadb-server					install
mariadb-server-10.3				install
mariadb-server-core-10.3			install

/.
/etc
/etc/mysql
/etc/mysql/mariadb.cnf
/etc/mysql/mariadb.conf.d
/usr
/usr/share
/usr/share/doc
/usr/share/doc/mariadb-common
/usr/share/doc/mariadb-common/changelog.Debian.gz
/usr/share/doc/mariadb-common/copyright

```

Closes PR #1690
Reviewed by: otto@kekalainen.net
2021-07-09 10:04:30 +02:00
Anel Husakovic
d2dddbff4e MDEV-26080: SHOW GRANTS does not quote role names properly for DEFAULT ROLE
- Used single quotes, back quotes are used with commit
fafb35ee51 in 10.3 and will be changed.

Reviewed by: serg@mariadb.org
2021-07-09 08:25:54 +02:00
Igor Babaev
83e442fc34 MDEV-26095 Infinite recursion when processing embedded recursive CTE
with missing RECURSIVE

If a table reference r used inthe specification of a CTE whose definition
is contained in the WITH clause where RECURSIVE is omitted then this table
reference cannot be considered as a recursive table reference even if it is
used in the query that specifies CTE whose name is r. It can be considered
only as a reference to an embedding CTE or to a temporary table or to
a base table/view. If there is no such object with name r then an error
message must be reported.
This patch fixes the code that actually in some cases resolved r as a
reference to the CTE whose specification contained r if its name was r
in spite of the fact that r was not considered as a recursive CTE.
This happened in the cases when the definition of r was used in the
specification of another CTE. Such wrong name resolution for r led to an
infinite recursive invocations of the parser that ultimately crashed the
server.
This bug is a result of the fix for mdev-13780 that was not quite correct.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2021-07-08 09:37:16 -07:00
Sergei Golubchik
621fae3cbc MDEV-25802 mtr: race condition if the test quickly restarts twice
part II

need to tell SafeProcess not to collect the exited mysqld
in sleep_until_file_created(), so that it would be found in the
later wait_any_timeout() in run_testcase()

Removed a strange condition in SafeProcess::wait_one()
that treated return value of -1 from waitpid() as "process exists"
instead of as "no such child process" (see `perldoc -f waitpid`)
2021-07-06 16:12:13 +02:00
Sergei Golubchik
1223cfe1d3 MDEV-25802 mtr: race condition if the test quickly restarts twice
expect file is always removed before starting a server.
So if it exists here, it means the server started successfully,
mysqltest continued executing the test, created a new expect file,
and shut down the server. All while we were waiting for the server
to start.

In other words, if the expect file exists, the server did actually start.
Even if it isn't running now.

This fixes occasional failures of innodb.log_corruption (in 10.6)
2021-07-06 16:12:13 +02:00
Sergei Golubchik
6a466db00a MDEV-25857 MTR should report at least last test that was executed in case of shutdown and not-completed
* return a success/failure value from mysqld_start()
  and don't error out / exit in mysqld_start(), the caller will do
* pass the correct $mysqld object into check_expected_crash_and_restart()
  instead of searching for it inside. Search in the caller
* so that when a failed restart changes $mysqld->{proc}, mtr would
  still detect it as a failed mysqld (by updating $proc to match)

also: log the server command line into the server error log
2021-07-06 16:12:13 +02:00
Aleksey Midenkov
22e4baaa5d MDEV-25595 DROP part of failed CREATE OR REPLACE is not written into binary log
Do log_drop_table() in case of failed mysql_prepare_create_table().
2021-07-06 00:47:41 +03:00
Sergei Petrunia
99f700a820 MDEV-25013: SIGSEGV in best_extension_by_limited_search | SIGSEGV in restore_prev_nj_state
Add testcase
2021-07-03 12:04:18 +03:00
Sergei Petrunia
d4177a7e09 MDEV-23937: SIGSEGV in looped best_extension_by_limited_search from greedy_search
Add a testcase (fixed by fix for MDEV-17783)
2021-07-03 12:04:08 +03:00
Sergei Petrunia
cdb29960d2 MDEV-17783: AddressSanitizer: stack-buffer-overflow in table_cond_selectivity
A less-intrusive fix: don't have table_cond_selectivity() assume that
there are less than MAX_REF_PARTS hash-join KEYUSEs.

If there are more than that, switch to using an array. Allocate the array
on the heap: we can't allocate it on MEM_ROOT as table_cond_selectivity()
is called many times during the optimization.

(Variant 2, with review input addressed)
2021-07-03 12:01:38 +03:00
Sergei Golubchik
7c02e8717d MDEV-26081 set role crashes when a hostname cannot be resolved
host can be NULL
2021-07-02 19:21:51 +02:00
Eugene Kosov
ffe744e77d submodules.cmake: add missing --depth=1 2021-07-02 13:25:20 +03:00
Marko Mäkelä
2bf6f2c054 MDEV-26077 Assertion err != DB_DUPLICATE_KEY or unexpected ER_TABLE_EXISTS_ERROR
This is a backport of 161e4bfafd.

trans_rollback_to_savepoint(): Only release metadata locks (MDL)
if the storage engines agree, after the changes were already rolled back.

Ever since commit 3792693f31
and mysql/mysql-server@55ceedbc3f
we used to cheat here and always release MDL if the binlog is disabled.

MDL are supposed to prevent race conditions between DML and DDL also
when no replication is in use. MDL are supposed to be a superset of
InnoDB table locks: InnoDB table lock may only exist if the thread
also holds MDL on the table name.

In the included test case, ROLLBACK TO SAVEPOINT would wrongly release
the MDL on both tables and let ALTER TABLE proceed, even though the DML
transaction is actually holding locks on the table.

Until commit 1bd681c8b3 (MDEV-25506)
in MariaDB 10.6, InnoDB would often work around the locking violation
in a blatantly non-ACID way: If locks exist on a table that is being
dropped (in this case, actually a partition of a table that is being
rebuilt by ALTER TABLE), InnoDB could move the table (or partition)
into a queue, to be dropped after the locks and references had been
released. If the lock is not released and the original copy of the
table not dropped quickly enough, a name conflict could occur on
a subsequent ALTER TABLE.

The scenario of commit 3792693f31
is unaffected by this fix, because mysqldump
would use non-locking reads, and the transaction would not be holding
any InnoDB locks during the execution of ROLLBACK TO SAVEPOINT.
MVCC reads inside InnoDB are only covered by MDL and page latches,
not by any table or record locks.

FIXME: It would be nice if storage engines were specifically asked
which MDL can be released, instead of only offering a choice
between all or nothing. InnoDB should be able to release any
locks for tables that are no longer in trx_t::mod_tables, except
if another transaction had converted some implicit record locks
to explicit ones, before the ROLLBACK TO SAVEPOINT had been completed.

Reviewed by: Sergei Golubchik
2021-07-02 11:15:35 +03:00
Marko Mäkelä
5a2b625843 MDEV-25129 fixup: Adjust test result
Fixup for commit 768c51880a
2021-07-02 11:08:48 +03:00
Daniel Black
c22f7f2323 MDEV-25129 postfix for windows
C:\projects\server\sql\sql_show.cc(7913): error C2220: warning treated as error - no 'object' file generated [C:\projects\server\win_build\sql\sql.vcxproj]
C:\projects\server\sql\sql_show.cc(7913): warning C4267: 'initializing': conversion from 'size_t' to 'uint', possible loss of data [C:\projects\server\win_build\sql\sql.vcxproj]

caused by 768c51880a
2021-07-02 15:58:13 +10:00
Sergei Petrunia
eb20c91b55 MDEV-25969: Condition pushdown into derived table doesn't work if select list uses SP
Consider a query of the form:

  select ... from (select item2 as COL1) as T where COL1=123

Condition pushdown into derived table will try to push "COL1=123" condition
down into table T.
The process of pushdown involves "substituting" the item, that is,
replacing Item_field("T.COL1") with its "producing item" item2.
In order to use item2, one needs to clone it (call Item::build_clone).

If the item is not cloneable (e.g. Item_func_sp is not), the pushdown
process will fail and nothing at all will be pushed.

Fixed by introducing transform_condition_or_part() which will try to apply
the transformation for as many parts of condition as possible. The parts of
condition that couldn't be transformed are dropped.
2021-06-30 13:52:23 +03:00
xing-zhi, jiang
768c51880a MDEV-25129 Add KEYWORDS view to the INFORMATION_SCHEMA
Add KEYWORDS table and SQL_FUNCTIONS table to INFORMATION_SCHEMA.
This commits needs some minor changes when propagated upwards
(e.g. func_array in item_create.cc has a termination element that
 doesn't exist in later versions of MariaDB)
2021-06-29 16:15:24 +03:00
Marko Mäkelä
58252fff15 MDEV-26040 os_file_set_size() may not work on O_DIRECT files
os_file_set_size(): Trim the current size down to the file system
block size, to obey the constraints for unbuffered I/O.
2021-06-29 14:28:23 +03:00
Alexey Botchkov
8147d2e618 MDEV-25461 Assertion `je->state == JST_KEY' failed in Geometry::create_from_json.
Handle invalid GEOJSON-s in Geometry::create_from_json().
2021-06-28 11:53:33 +04:00
Igor Babaev
4e4f742ed7 Adjusted test results after the fix for MDEV-20411 (2) 2021-06-26 23:11:10 -07:00
Igor Babaev
8b3f816cab Adjusted test results after the fix for MDEV-20411 2021-06-26 08:51:17 -07:00
Igor Babaev
12c80df482 MDEV-20411 Procedure containing CTE incorrectly stored in mysql.proc
If the first token of the body of a stored procedure was 'WITH' then
the beginning of the body was determined incorrectly and that token was
missing in the string representing the body of the SP in mysql.proc. As a
resultnany call of such procedure failed as the string representing the
body could not be parsed.

The patch corrects the code of the functions get_tok_start() and
get_cpp_tok_start() of the class Lex_input_stream to make them take into
account look ahead tokens. The patch is needed only for 10.2 as this
problem has neen resolved in 10.3+.
2021-06-25 18:06:08 -07:00
Olivier Bertrand
1c4b917f0e Fix clang compile error in value.cpp 2021-06-25 18:35:26 +02:00
Olivier Bertrand
7863554e42 Fix clang compile error in tabjson and typo in tabbson.cpp 2021-06-25 12:54:34 +02:00
Olivier Bertrand
af7a0895ac Fix clang compile error in tabjson.cpp and tabbson.cpp 2021-06-25 12:30:10 +02:00
Olivier Bertrand
6239e2a4ec Fix clang compile error in tabjson.cpp and tabbson.cpp 2021-06-25 11:38:16 +02:00
Julius Goryavsky
4ad148b148 MDEV-26019: Upgrading MariaDB breaks TLS mariabackup SST
Fixed typo in variable name that breaks SST in some scenatios.

Also fixed one small inaccuracy after MDEV-25978 which leads
to the use of an uninitialized variable when the --log-bin
option is specified without an argument.
2021-06-25 07:09:54 +02:00
Olivier Bertrand
5c5d60b897 Fix compile error in value.h and remove unused variables in ha_connect.cc 2021-06-25 01:15:51 +02:00
Olivier Bertrand
330b32ebd9 Fix clang compile error 2021-06-24 23:56:13 +02:00
Olivier Bertrand
ed70f76cf7 - Make function strz return null when LEX_STRING is null
modified:   storage/connect/ha_connect.cc

- Use NOTE instead of WARNING in connect_assisted_discovery
  This because MariaDB raise an error when doing so
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/tabrest.cpp

- Make MONGO tables recognize STRINGIFY and JsonAllPath
  modified:   storage/connect/mongo.cpp
  modified:   storage/connect/mongo.h
  modified:   storage/connect/tabcmg.h
  modified:   storage/connect/tabjmg.cpp
  modified:   storage/connect/tabcmg.cpp
  modified:   storage/connect/tabjmg.h

- Fix OBJECT option for Pretty != 2 JSN and BSON tables
  Accept all syntaxes  for the OBJECT option
  modified:   storage/connect/tabbson.cpp
  modified:   storage/connect/tabjson.cpp

- Use my_snprintf in function xcurl (by vuvova)
  modified:   storage/connect/tabrest.cpp

- Format dates entered as integer when formatted
  modified:   storage/connect/value.cpp
  modified:   storage/connect/value.h
2021-06-24 00:46:12 +02:00
Jan Lindström
9258cfa4b4 MDEV-25978 : rsync SST does not work with custom binlog name
wsrep_sst_common did not correctly set name for binlog index
file if custom binlog name was used and this name was
not added to script command line.

Added test case for both log_basename and log_binlog.
2021-06-23 08:11:05 +03:00
Jan Lindström
83464029ce Fix try for Galera test lp1376747-4 2021-06-23 08:10:35 +03:00
Daniel Bartholomew
f67aee000d
bump the VERSION 2021-06-22 22:00:23 -04:00