Commit graph

189365 commits

Author SHA1 Message Date
Sergei Petrunia
3c209bfc04 MDEV-25994: Crash with union of my_decimal type in ORDER BY clause
When single-row subquery fails with "Subquery reutrns more than 1 row"
error, it will raise an error and return NULL.

On the other hand, Item_singlerow_subselect sets item->maybe_null=0
for table-less subqueries like "(SELECT not_null_value)"  (*)

This discrepancy (item with maybe_null=0 returning NULL) causes the
code in Type_handler_decimal_result::make_sort_key_part() to crash.

Fixed this by allowing inference (*) only when the subquery is NOT a
UNION.
2022-04-22 13:57:16 +03:00
Sergei Golubchik
4d11290065 .gitignore 2022-04-21 10:06:01 +02:00
Sergei Golubchik
6f6c74b0d1 Merge branch '10.2' into 10.3 2022-04-21 10:05:50 +02:00
Marko Mäkelä
4730314a70 MDEV-28369 ibuf_bitmap_mutex is an unnecessary contention point
The only purpose of ibuf_bitmap_mutex is to prevent a deadlock between
two concurrent invocations of ibuf_update_free_bits_for_two_pages_low()
on the same pair of bitmap pages, but in opposite order.
The mutex is unnecessarily serializing the execution of the function
even when it is being invoked on totally different tablespaces.
To avoid deadlocks, it suffices to ensure that the two page latches
are being acquired in a deterministic (sorted) order.
2022-04-21 09:15:18 +03:00
Alexander Barkov
2be617d869 MDEV-25243 ASAN heap-use-after-free in Item_func_sp::execute_impl upon concurrent view DDL and I_S query with view and function 2022-04-21 09:51:11 +04:00
Sergei Golubchik
5ba77222e9 MDEV-21028 Server crashes in Query_arena::set_query_arena upon SELECT from view
if the view has algorithm=temptable it is not updatable,
so DEFAULT() for its fields is meaningless,
and thus it's NULL or 0/'' for NOT NULL columns.
2022-04-20 22:07:48 +02:00
Thirunarayanan Balathandayuthapani
372b0e6355 MDEV-20194 Warnings inconsistently issued upon CHECK on table from older versions
The following condition has to added:
1) InnoDB fails to include the offset of the node pointer field
in non-leaf record for redundant row format.

2) If the Fixed length field does have only prefix length then
calculate the field maximum size as prefix length.

- Added the test case to test (2) and to check maximum number of
fields can exist in the index.
2022-04-20 19:55:17 +05:30
Jan Lindström
84b135065c MDEV-28314 : The Galera cluster primary node goes into hang mode when innodb_encryption_threads is enabled
When we enable writes after Galera SST srv_n_fil_crypt_threads needs
to be set temporally to 0 (as was done when writes were disabled)
to make sure that encryption threads will be really started based
on old value of encryption threads.

Fix provided by Marko Mäkelä.
2022-04-20 07:42:21 +03:00
Sergei Golubchik
9c5fd0f624 vcols: cannot use CONTEXT_ANALYSIS_ONLY_VCOL_EXPR on fix_fields
because CONTEXT_ANALYSIS_ONLY_VCOL_EXPR can be used only for,
exactly, context analysys. Items fixed that way cannot be evaluated.
But vcols are going to be evaluated, so they have to be fixed properly,
for evaluation.
2022-04-19 17:10:05 +02:00
Sergei Golubchik
a59f483c06 MDEV-28092 MariaDB SEGV issue
add test
2022-04-19 17:10:05 +02:00
Marko Mäkelä
5aef0123a7 MDEV-28317 Assertion failures in row_undo_mod on recovery
Starting with 10.3, an assertion would fail on the rollback of
a recovered incomplete transaction if a table definition violates
a FOREIGN KEY constraint.

DICT_ERR_IGNORE_RECOVER_LOCK: Include also DICT_ERR_IGNORE_FK_NOKEY
so that trx_resurrect_table_locks() will be able to load
table definitions and resurrect IX locks. Previously, if the
FOREIGN KEY constraints of a table were incomplete, the table
would fail to load until rollback, and in 10.3 or later an assertion
would fail that the rollback was not protected by a table IX lock.

Thanks to commit 9de2e60d74 there
will be no problems to enforce subsequent FOREIGN KEY operations
even though a table with invalid REFERENCES clause was loaded.
2022-04-19 12:40:05 +03:00
Marko Mäkelä
b3c3291f0b MDEV-24176 fixup: GCC -Wmaybe-uninitialized 2022-04-19 10:29:21 +03:00
Aleksey Midenkov
08c7ab404f MDEV-24176 Server crashes after insert in the table with virtual
column generated using date_format() and if()

vcol_info->expr is allocated on expr_arena at parsing stage. Since
expr item is allocated on expr_arena all its containee items must be
allocated on expr_arena too. Otherwise fix_session_expr() will
encounter prematurely freed item.

When table is reopened from cache vcol_info contains stale
expression. We refresh expression via TABLE::vcol_fix_exprs() but
first we must prepare a proper context (Vcol_expr_context) which meets
some requirements:

1. As noted above expr update must be done on expr_arena as there may
be new items created. It was a bug in fix_session_expr_for_read() and
was just not reproduced because of no second refix. Now refix is done
for more cases so it does reproduce. Tests affected: vcol.binlog

2. Also name resolution context must be narrowed to the single table.
Tested by: vcol.update main.default vcol.vcol_syntax gcol.gcol_bugfixes

3. sql_mode must be clean and not fail expr update.

sql_mode such as MODE_NO_BACKSLASH_ESCAPES, MODE_NO_ZERO_IN_DATE, etc
must not affect vcol expression update. If the table was created
successfully any further evaluation must not fail. Tests affected:
main.func_like

Reviewed by: Sergei Golubchik <serg@mariadb.org>
2022-04-18 12:44:27 +03:00
Aleksey Midenkov
c02ebf3510 MDEV-24176 Preparations
1. moved fix_vcol_exprs() call to open_table()

mysql_alter_table() doesn't do lock_tables() so it cannot win from
fix_vcol_exprs() from there. Tests affected: main.default_session

2. Vanilla cleanups and comments.
2022-04-18 12:44:27 +03:00
Oleg Smirnov
7498978e6a MDEV-27699 ANALYZE FORMAT=JSON fields are incorrect for UNION ALL queries
UNION ALL queries are a subject of optimization introduced in MDEV-334
when creation of a temporary table is skipped.
While there is a check for this optimization in Explain_union::print_explain()
there was no such in Explain_union::print_explain_json(). This resulted in
printing irrelevant data like:
  "union_result": {
    "table_name": "<union2,3>",
    "access_type": "ALL",
    "r_loops": 0,
    "r_rows": null
in case when creation of the temporary table was actually optimized out.
This commits adds a check whether the temporary table was actually created
during the UNION ALL processing and eliminates printing of the irrelevant data.
2022-04-18 07:50:14 +03:00
Oleksandr Byelkin
e4e25d2bac MDEV-26423 MariaDB server crash in Create_tmp_table::finalize
Removed prohibition of creating temporary field of Item_default_value
(added by mistake by 1d9b043a1f fix of
MDEV-10780 and MDEV-11265).
2022-04-15 16:58:11 +02:00
Sergei Golubchik
c274853c07 MDEV-25638 Assertion `!result' failed in convert_const_to_int
When fixing vcols, fix_fields might call convert_const_to_int().
And that will try to read the field value (from record[0]).
Mark the table as having no data to prevent that, because record[0]
is not initialized yet.
2022-04-15 00:25:42 +02:00
Sergei Golubchik
4681b6f2d8 MDEV-26281 ASAN use-after-poison when complex conversion is involved in blob
the bug was that in_vector array in Item_func_in was allocated in the
statement arena, not in the table->expr_arena.

revert part of the 5acd391e8b. Instead, change the arena correctly
in fix_all_session_vcol_exprs().

Remove TABLE_ARENA, that was introduced in 5acd391e8b to force
item tree changes to be rolled back (because they were allocated in the
wrong arena and didn't persist. now they do)
2022-04-14 21:45:20 +02:00
Sergei Golubchik
cc08c43ed6 cleanup: remove Item_default_value::cached_field 2022-04-14 15:43:35 +02:00
Sergei Golubchik
b5e16a6e03 MDEV-26061 MariaDB server crash at Field::set_default
* Item_default_value::fix_fields creates a copy of its argument's field.
* Field::default_value is changed when its expression is prepared in
  unpack_vcol_info_from_frm()

This means we must unpack any vcol expression that includes DEFAULT(x)
strictly after unpacking x->default_value.

To avoid building and solving this dependency graph on every table open,
we update Item_default_value::field->default_value after all vcols
are unpacked and fixed.
2022-04-14 15:43:35 +02:00
Alexander Barkov
c05fd70097 MDEV-26323 use-after-poison issue of MariaDB server 2022-04-14 17:12:30 +04:00
Daniel Black
66832e3a64 mtr: extend gdb backtace info
bt full - to include args and locals.

set print sevenbit on
  - it is more useful to be able to see the exact bytes
    (in case something is dumped as a string and not hexadecimal digits)
set print static-members off
  - there are many interesting (non-const) static members
set frame-arguments all
  - even non-printables are useful to see.

Let's make our bb logs give a little bit more detail on those
hard to reproduce bugs.

Tests on rhel7's gdb-7.6.1-120.el7
2022-04-14 19:55:26 +10:00
Alexander Barkov
83516a33a8 An additional patch for MDEV-27690 Crash on CHARACTER SET csname COLLATE DEFAULT in column definition
Applying the fix to sql_yacc_ora.yy. Adding a test for sql_mode=ORACLE.
2022-04-14 12:22:28 +04:00
Alexander Barkov
9d734cdd61 Merge remote-tracking branch 'origin/10.2' into 10.3 2022-04-14 11:50:34 +04:00
Oleksandr Byelkin
f130a5ea3c MDEV-28266 Crash in Field_string::type_handler when calling procedures
on_table_fill_finished() should always be done at the end of open()
even if result is not Select_materialize but (for example)
Select_fetch_into_spvars.
2022-04-13 15:37:46 +02:00
Marko Mäkelä
e4835bf572 MDEV-28304 innodb.instant_alter,8k.rdiff does not apply on FreeBSD
This is conceptually a backport of
commit 6f4740fde7
(re-recording the .rdiff files).
2022-04-13 09:23:52 +03:00
Julius Goryavsky
42908dc5fb MDEV-26171: wsrep_sst_receive_address does not parse IPv6 address correctly
This commit fixes problems with parsing ipv6 addresses given via
the wsrep_sst_receive_address and wsrep_node_address options.

Also, this commit removes extra lines in the configuration files
in the mtr test suites for Galera related to these parameters.
2022-04-12 17:14:39 +02:00
Sergei Golubchik
d31732e02e MDEV-27834 Incorrect provides of MariaDB-shared for CentOS8
fix bad merge
2022-04-12 13:38:41 +02:00
Sergei Golubchik
bbdec04d59 MDEV-24317 Data race in LOGGER::init_error_log at sql/log.cc:1443 and in LOGGER::error_log_print at sql/log.cc:1181
don't initialize error_log_handler_list in set_handlers()
* error_log_handler_list is initialized to LOG_FILE early, in init_base()
* set_handlers always reinitializes it to LOG_FILE, so it's pointless
* after init_base() concurrent threads start using sql_log_warning,
  so following set_handlers() shouldn't modify error_log_handler_list
  without some protection
2022-04-12 13:07:20 +02:00
Shunpoco
767d8d8335 MDEV-27448 MTR returns success (zero) upon invalid option
I change from `exit;` to `exit(1);` on a function `usage()`.
When we try to run mtr with a wrong option, a function `usage()` is called with the wrong option as its argument. In this case, because the function call `exit` in a first if statement, we get exit status 0.
2022-04-12 08:17:32 +10:00
Alexander Barkov
833f4486cf MDEV-27690 Crash on CHARACTER SET csname COLLATE DEFAULT in column definition
This is a temporary fix for 10.2.
This problem was permanently fixed in 10.9 under terms of MDEV-27743.

This patch should propagate up to 10.8 then null-merged to 10.9.
2022-04-11 19:45:31 +04:00
Monty
6891c4874a MDEV-28269 Assertion `save_errno' in maria_write or ER_GET_ERRNO
The issue was that the value of MARIA_FOUND_WRONG_KEY was a value
that could be returned by ha_key_cmp.

This was already fixed in MyISAM, now using the same fix in Aria:
Setting the value to INT_MAX32, which should be impossible in any
normal cases.

I also fixed so that if there is a wrong key, we now get a proper error
message and not an assert.
2022-04-11 17:30:28 +03:00
KiyoshiTakeda
4d1955d348
MDEV-28225 Disallow user to create Spider temporary table
Creating a temporary table with Spider is non-sense because a Spider
table cannot hold any physical data and it requires an additional
effort to manage even if it is configured correctly.

Set HTON_TEMPORARY_NOT_SUPPORTED to spider_hton->flags.  

Reviewed-by: nayuta.yanagisawa@hey.com
Co-authored-by: d8sk4ueun@gmail.com
2022-04-11 23:02:38 +09:00
Alexander Barkov
2ae92e8981 MDEV-28267 ASAN heap-use-after-free in Item_sp::func_name_cstring
This crash happens on a combination of multiple conditions:

- There is a thead#1 running an "ANALYZE FORMAT=JSON" query for a
  "SELECT .. FROM INFORMATION_SCHEMA.COLUMNS WHERE .. "
- The WHERE clause contains a stored function call, say f1().
- The WHERE clause is built in the way so that the function f1()
  is never actually called, e.g.
    WHERE .. AND (TRUE OR f1()=expr)
- The database contains multiple VIEWs that have the function f1() call,
  e.g. in their <select list>
- The WHERE clause is built in the way so that these VIEWs match
  the condition.
- There is a parallel thread#2 running. It creates or drops or recreates
  some other stored routine, say f2(), which is not used in the ANALYZE query.
  It effectively invalidates the stored routine cache for thread#1
  without locking.
  Note, it is important that f2() is NOT used by ANALYZE query.
  Otherwise, thread#2 would be locked until the ANALYZE query
  finishes.

When all of the above conditions are met, the following happens:

1. thread#1 starts the ANALYZE query. It notices a call for the stored function
   f1() in the WHERE condition. The function f1() gets parsed and cached
   to the SP cache. Its address also gets assigned to Item_func_sp::m_sp.

2. thread#1 starts iterating through all tables that
   match the WHERE condition to find the information about their columns.

3. thread#1 processes columns of the VIEW v1.
   It notices a call for f1() in the VIEW v1 definition.
   But f1() is already cached in the step#1 and it is up to date.
   So nothing happens with the SP cache.

4. thread#2 re-creates f2() in a non-locking mode.
   It effectively invalidates the SP cache in thread#1.

5. thread#1 processes columns of the VIEW v2.
   It notices a call for f1() in the VIEW v2 definition.
   It also notices that the cached version of f1() is not up to date.
   It frees the old definition of f1(), parses it again, and puts a
   new version of f1() to the SP cache.

6. thread#1 finishes processing rows and generates the JSON output.
   When printing the "attached_condition" value, it calls
   Item_func_sp::print() for f1(). But this Item_func_sp links
   to the old (freed) version of f1().

The above scenario demonstrates that Item_func_sp::m_sp can point to an
alredy freed instance when Item_func_sp::func_name() is called,
so accessing to Item_sp::m_sp->m_handler is not safe.

This patch rewrites the code to use Item_func_sp::m_handler instead,
which is always reliable.

Note, this patch is only a cleanup for MDEV-28166 to quickly fix the regression.
It fixes MDEV-28267. But it does not fix the core problem:
The code behind I_S does not take into account that the SP
cache can be updated while evaluating rows of the COLUMNS table.
This is a corner case and it never happens with any other tables.
I_S.COLUMNS is very special.

Another example of the core problem is reported in MDEV-25243.
The code accesses to Item_sp::m_sp->m_chistics of an
already freed m_sp, again. It will be addressed separately.
2022-04-09 23:01:26 +04:00
Sergei Golubchik
d623b5a1dd MDEV-22282 When using mysqldump to backup a view that contains derived tables, the database name is prepended to each table in the view
derived tables have db = "", table_name = "*", those aren't real names
to be compared with.
2022-04-09 11:49:11 +02:00
Nayuta Yanagisawa
27b5d814e2 MDEV-27065 Partitioning tables with custom data directories moves data back to default directory
The partitioning engine does not support the table-level DATA/INDEX
DIRECTORY specification.

If one create a non-partitioned table with the DATA/INDEX DIRECTORY
option and then performs ALTER TABLE ... PARTITION BY on it, the
DATA/INDEX DIRECTORY specification of the old schema is ignored.

The behavior might be a bit surprising for users because the value
of a usual table option applies to all the partitions. Thus, we raise
a warning on such ALTER TABLE ... PARTITION BY.
2022-04-08 16:49:10 +09:00
Alexander Barkov
3814b04d6b MDEV-28062 Assertion `(length % 4) == 0' failed in my_lengthsp_utf32 on INSERT..SELECT
Adding an MTR test only.

This problem was earlier fixed by the patch for:
  MDEV-28078 Garbage on multiple equal ENUMs with tricky character sets
2022-04-08 11:36:31 +04:00
Nayuta Yanagisawa
4194f7b605 MDEV-25116 Spider: IF(COUNT( trigger SQL Error (1054)_ Unknown column '' in field list
The original query "SELECT IF(COUNT(a.`id`)>=0,'Y','N') FROM t" is
transformed to "SELECT COUNT(a.`id`), IF(ref >= 0, 'Y', 'N') FROM t",
where ref is Item_ref to "COUNT(a.`id`)", by split_sum_func().

Spider walks the item list twice, invoking spider_db_print_item_type().
The first invocation is in spider_create_group_by_handler() with
str == NULL. The second one is in spider_group_by_handler::init_scan()
with str != NULL.

spider_db_print_item_type() prints nothing at the first invocation,
and it prints item at the second invocation. However, at the second
invocation, the above mentioned ref to "COUNT(a.`id`)" points to
a field in a temporary table where the result will be stored. Thus,
to look behind the item_ref, Spider need to generate the query earlier.

A possible fix would be to generate a query to send in
spider_create_group_by_handler(). However, the fix requires a
considerable amount of changes of the Spider's GROUP BY handler.
I'd like to avoid that.

So, I fix the problem by not to use the GROUP BY handler when a
query contains Item_ref whose table_name, name, and alias_name_used
are not set.
2022-04-08 15:27:33 +09:00
Sergei Golubchik
b725a91757 MDEV-28253 Mysqldump - INVISIBLE column error 2022-04-07 23:02:23 +02:00
Sergei Petrunia
5a8766a980 Better comments in Item_in_subselect::inject_in_to_exists_cond() 2022-04-07 15:40:34 +03:00
Julius Goryavsky
7a03128faf MDEV-28205: SST via mariabackup stops on failure while archiving logs
Currenly SST script for mariabackup stops on any failure while archiving
logs, e.g. when unable to create directory, insufficient permissions, gzip
failure, etc. However, in case of such problems, the script should issue
a warning and continue without archiving, but not exit with a fatal error.

This commit adds this fix to the SST script for mariabackup.
2022-04-07 13:26:36 +02:00
Jan Lindström
3c99a48db3 MDEV-28247 : Disable background ibuf merge during Galera SST
This failure was caused by MDEV-25975, which removed the parameter
innodb_disallow_writes.

Added a check for wsrep_sst_disable_writes to the function
ibuf_merge_in_background().
2022-04-07 08:45:01 +03:00
Alexander Barkov
7355f7b1f5 Adding MTR tests to cover how keywords of different kinds behave in various contexts 2022-04-07 06:13:22 +04:00
Oleg Smirnov
53b580a91c MDEV-28077 'Wrong create options' error with 'big_tables' enabled
The cause of the bug is overflow of uint16 KEY_PART_INFO::length and/or
uint16 KEY_PART_INFO::store_length. The solution is to increase the size
of those variables to the 'uint' type (which is 32-bit long)
2022-04-06 15:28:34 +03:00
Oleg Smirnov
85192553ae MDEV-24560 SIGSEGV in st_join_table::cleanup
If JOIN::create_postjoin_aggr_table encounters errors during execution
then free_tmp_table() is then called twice for JOIN_TAB::aggr.
The solution is to initialize JOIN_TAB::aggr only on successful completion
of JOIN::create_postjoin_aggr_table
2022-04-06 15:12:07 +03:00
Marko Mäkelä
e9735a8185 MDEV-25975 innodb_disallow_writes causes shutdown to hang
We will remove the parameter innodb_disallow_writes because it is badly
designed and implemented. The parameter was never allowed at startup.
It was only internally used by Galera snapshot transfer.
If a user executed
SET GLOBAL innodb_disallow_writes=ON;
the server could hang even on subsequent read operations.

During Galera snapshot transfer, we will block writes
to implement an rsync friendly snapshot, as follows:

sst_flush_tables() will acquire a global lock by executing
FLUSH TABLES WITH READ LOCK, which will block any writes
at the high level.

sst_disable_innodb_writes(), invoked via ha_disable_internal_writes(true),
will suspend or disable InnoDB background tasks or threads that could
initiate writes. As part of this, log_make_checkpoint() will be invoked
to ensure that anything in the InnoDB buf_pool.flush_list will be written
to the data files. This has the nice side effect that the Galera joiner
will avoid crash recovery.

The changes to sql/wsrep.cc and to the tests are based on a prototype
that was developed by Jan Lindström.

Reviewed by: Jan Lindström
2022-04-06 08:06:49 +03:00
Marko Mäkelä
7c584d8270 Merge 10.2 into 10.3 2022-04-06 08:06:35 +03:00
Sergei Golubchik
2d2c3da8ec MDEV-27673 Warning after "select progress from information_schema.processlist"
after moving fields in optimize_schema_tables_memory_usage()
store default values into their new, moved, locations.
2022-04-05 13:09:44 +02:00
Sergei Golubchik
cf8d30efd2 Revert "MDEV-28131 Unexpected warning while selecting from information_schema.processlist"
This reverts commit 0812d0de8d.
But keeps the test case.
2022-04-05 13:09:44 +02:00
Sergei Golubchik
d7fd76456e MDEV-19525 fix the test for embedded
followup for 58cd2a8ded
2022-04-05 13:09:44 +02:00