Commit graph

195664 commits

Author SHA1 Message Date
Marko Mäkelä
aa3a9d1ef5 Merge 10.6 into 10.7 2022-04-12 16:11:29 +03:00
Marko Mäkelä
cdc0bbdd87 Merge 10.5 into 10.6 2022-04-12 14:53:58 +03:00
Sergei Golubchik
d31732e02e MDEV-27834 Incorrect provides of MariaDB-shared for CentOS8
fix bad merge
2022-04-12 13:38:41 +02:00
Sergei Golubchik
bbdec04d59 MDEV-24317 Data race in LOGGER::init_error_log at sql/log.cc:1443 and in LOGGER::error_log_print at sql/log.cc:1181
don't initialize error_log_handler_list in set_handlers()
* error_log_handler_list is initialized to LOG_FILE early, in init_base()
* set_handlers always reinitializes it to LOG_FILE, so it's pointless
* after init_base() concurrent threads start using sql_log_warning,
  so following set_handlers() shouldn't modify error_log_handler_list
  without some protection
2022-04-12 13:07:20 +02:00
Alexander Barkov
e41500e4f2 MDEV-26128 type_set and type_enum are broken
The problem was not with the server behavior,
it was with wrong test results recorded.

Enabling both type_set and type_enum.

type_set.result was OK. The test did not fail.

type_enum.result was incorrectly recorded in this commit:
  eb483c5181

Recording correct results.
2022-04-12 14:26:02 +04:00
Thirunarayanan Balathandayuthapani
011b332d81 MDEV-28037 Assertion `trx->bulk_insert' failed in innodb_prepare_commit_versioned
- Trigger statement initiate update statement after bulk insert
operation and leads to disable of bulk operation. During commit,
InnoDB expects transaction to be in bulk insert mode before
applying the bulk insert operation. InnoDB transaction should
apply all bulk insert operation before disabling bulk insert
operation.
2022-04-12 14:41:11 +05:30
Monty
ef8d203a31 Added scripts/wsrep_sst_backup to .gitignore 2022-04-12 11:33:52 +03:00
Monty
22f7190e85 MDEV-28074 mysqldump --order-by-size
Author: Erki Aring
2022-04-12 11:32:02 +03:00
Marko Mäkelä
c5512878fe MDEV-14425 fixup: Fix a typo in mariadb-backup 2022-04-12 10:27:18 +03:00
Marko Mäkelä
ca3bbf4c0c Merge 10.5 into 10.6 2022-04-12 09:26:02 +03:00
Tuukka Pasanen
bc75f7ed6d MDEV-28194: Remove unneeded path from MariaDB server postinst script
Remove unndeeded paths from mariadb-server-10.6.postinst as
it's already in export PATH and it fires 'command-with-path-in-maintainer-script'
in lintian checks
2022-04-12 13:28:30 +10:00
Shunpoco
767d8d8335 MDEV-27448 MTR returns success (zero) upon invalid option
I change from `exit;` to `exit(1);` on a function `usage()`.
When we try to run mtr with a wrong option, a function `usage()` is called with the wrong option as its argument. In this case, because the function call `exit` in a first if statement, we get exit status 0.
2022-04-12 08:17:32 +10:00
Tuukka Pasanen
e858fc89d3 MDEV-27033: Clean lintian 'duplicate-override-context' warnings
Fix Debian lintian warnings:
  W: mariadb-10.7 source: duplicate-override-context version-substvar-for-external-package (lines 12 13) [debian/source/lintian-overrides:13]
  W: mariadb-10.7 source: duplicate-override-context version-substvar-for-external-package (lines 14 15) [debian/source/lintian-overrides:15]
  W: mariadb-10.7 source: duplicate-override-context version-substvar-for-external-package (lines 16 17) [debian/source/lintian-overrides:17]
  W: mariadb-10.7 source: duplicate-override-context version-substvar-for-external-package (lines 18 19) [debian/source/lintian-overrides:19]
  W: mariadb-10.7 source: duplicate-override-context version-substvar-for-external-package (lines 20 21) [debian/source/lintian-overrides:21]
  W: mariadb-10.7 source: duplicate-override-context version-substvar-for-external-package (lines 4 5) [debian/source/lintian-overrides:5]
by removing duplicate override lines
2022-04-12 08:16:09 +10:00
Rucha Deodhar
3327bb6098 MDEV-22266: Diagnostics_area::sql_errno() const: Assertion
`m_status == DA_ERROR' failed on SELECT after setting tmp_disk_table_size.

Analysis: Mismatch in number of warnings between "194 warnings" vs
"64 rows in set" is because of max_error_count variable which has default
value of 64.
About the corrupted tables, the error that occurs because of insufficient
tmp_disk_table_size variable is not reported correctly and we continue to
execute the statement. But because the previous error (about table being
full)is not reported correctly, this error moves up the stack and is
wrongly reported as parsing error later on while parsing frm file of one
of the information schema table. This parsing error gives corrupted table
error.
As for the innodb error, it occurs even when tmp_disk_table_size is not
insufficient is default but the internal error handler takes care of it
and the error doesn't show. But when tmp_disk_table_size is insufficient,
the fatal error which wasn't reported correctly moves up the stack so
internal error handler is not called. So it shows errors.
Fix: Report the error correctly.
2022-04-12 01:22:51 +05:30
Alexander Barkov
833f4486cf MDEV-27690 Crash on CHARACTER SET csname COLLATE DEFAULT in column definition
This is a temporary fix for 10.2.
This problem was permanently fixed in 10.9 under terms of MDEV-27743.

This patch should propagate up to 10.8 then null-merged to 10.9.
2022-04-11 19:45:31 +04:00
Monty
6891c4874a MDEV-28269 Assertion `save_errno' in maria_write or ER_GET_ERRNO
The issue was that the value of MARIA_FOUND_WRONG_KEY was a value
that could be returned by ha_key_cmp.

This was already fixed in MyISAM, now using the same fix in Aria:
Setting the value to INT_MAX32, which should be impossible in any
normal cases.

I also fixed so that if there is a wrong key, we now get a proper error
message and not an assert.
2022-04-11 17:30:28 +03:00
Thirunarayanan Balathandayuthapani
f61f58e759 MDEV-26578 ERROR: AddressSanitizer: heap-use-after-free around dict_table_t::is_temporary_name
- Purge thread is trying to access the table name when other thread
does rename of the table name. It leads to heap use after free error
by purge thread. purge thread should check whether the table name
is temporary while holding dict_sys.mutex.
2022-04-11 19:59:01 +05:30
KiyoshiTakeda
4d1955d348
MDEV-28225 Disallow user to create Spider temporary table
Creating a temporary table with Spider is non-sense because a Spider
table cannot hold any physical data and it requires an additional
effort to manage even if it is configured correctly.

Set HTON_TEMPORARY_NOT_SUPPORTED to spider_hton->flags.  

Reviewed-by: nayuta.yanagisawa@hey.com
Co-authored-by: d8sk4ueun@gmail.com
2022-04-11 23:02:38 +09:00
Marko Mäkelä
840bab858b MDEV-28289 fts_optimize_sync_table() is acquiring dict_sys.latch while holding it
dict_acquire_mdl_shared(): Invoke the correct variant
dict_table_t::parse_name<true>() also when trylock=true,
that is, we are being called from fts_optimize_sync_table().

Ever since commit 82b7c561b7 (MDEV-24258)
this code was prone to a hang. If another thread requested an
exclusive dict_sys.latch between the time
dict_acquire_mdl_shared<trylock=true>() acquired a shared dict_sys.latch
and dict_table_t::parse_name<false>() was trying to acquire another
shared dict_sys.latch, InnoDB would get into an infinite livelock
of threads, because the futex-based srw-lock implementation prioritizes
exclusive latch requests.

dict_table_t::parse_name(): Rename the template parameter dict_locked
into dict_frozen.
2022-04-11 14:41:24 +03:00
Marko Mäkelä
7bccf3dd74 MDEV-28274 Assertion s <= READ_FIX failed in buf_page_t::set_state
buf_page_t::set_state(): Relax a debug assertion. It is fine to update
a read-fixed block descriptor to be both read-fixed and buffer-fixed.

buf_pool_t::watch_unset(): Fix some incorrect logic that was implemented
in commit e9e6db9355.

Thanks to Elena Stepanova for the test case.
2022-04-11 10:22:40 +03:00
Vladislav Vaintroub
5a4a37076d MDEV-10183 implement service_manager_extend_timeout on Windows
The implementation calls SetServiceStatus() with updated
SERVICE_STATUS::dwHint and SERVICE_STATUS::dwCheckpoint
2022-04-11 07:49:43 +02:00
Alexander Barkov
bf70532e3d 10.5 tests for MDEV-26507 Assertion `tmp != ((long long) 0x8000000000000000LL)' failed in TIME_from_longlong_datetime_packed
The fix for MDEV-27673 (in 10.3) fixed MDEV-26507 as well.
This patch only adds MTR tests.
2022-04-10 15:28:35 +04:00
Alexander Barkov
284ff64cd6 10.6 tests for MDEV-26507 Assertion `tmp != ((long long) 0x8000000000000000LL)' failed in TIME_from_longlong_datetime_packed
The patch for MDEV-27673 (in 10.3) fixed MDEV-26507 as well.

This patch only adds 10.6 specific MTR tests for MDEV-26507.
2022-04-10 14:46:24 +04:00
Alexander Barkov
2ae92e8981 MDEV-28267 ASAN heap-use-after-free in Item_sp::func_name_cstring
This crash happens on a combination of multiple conditions:

- There is a thead#1 running an "ANALYZE FORMAT=JSON" query for a
  "SELECT .. FROM INFORMATION_SCHEMA.COLUMNS WHERE .. "
- The WHERE clause contains a stored function call, say f1().
- The WHERE clause is built in the way so that the function f1()
  is never actually called, e.g.
    WHERE .. AND (TRUE OR f1()=expr)
- The database contains multiple VIEWs that have the function f1() call,
  e.g. in their <select list>
- The WHERE clause is built in the way so that these VIEWs match
  the condition.
- There is a parallel thread#2 running. It creates or drops or recreates
  some other stored routine, say f2(), which is not used in the ANALYZE query.
  It effectively invalidates the stored routine cache for thread#1
  without locking.
  Note, it is important that f2() is NOT used by ANALYZE query.
  Otherwise, thread#2 would be locked until the ANALYZE query
  finishes.

When all of the above conditions are met, the following happens:

1. thread#1 starts the ANALYZE query. It notices a call for the stored function
   f1() in the WHERE condition. The function f1() gets parsed and cached
   to the SP cache. Its address also gets assigned to Item_func_sp::m_sp.

2. thread#1 starts iterating through all tables that
   match the WHERE condition to find the information about their columns.

3. thread#1 processes columns of the VIEW v1.
   It notices a call for f1() in the VIEW v1 definition.
   But f1() is already cached in the step#1 and it is up to date.
   So nothing happens with the SP cache.

4. thread#2 re-creates f2() in a non-locking mode.
   It effectively invalidates the SP cache in thread#1.

5. thread#1 processes columns of the VIEW v2.
   It notices a call for f1() in the VIEW v2 definition.
   It also notices that the cached version of f1() is not up to date.
   It frees the old definition of f1(), parses it again, and puts a
   new version of f1() to the SP cache.

6. thread#1 finishes processing rows and generates the JSON output.
   When printing the "attached_condition" value, it calls
   Item_func_sp::print() for f1(). But this Item_func_sp links
   to the old (freed) version of f1().

The above scenario demonstrates that Item_func_sp::m_sp can point to an
alredy freed instance when Item_func_sp::func_name() is called,
so accessing to Item_sp::m_sp->m_handler is not safe.

This patch rewrites the code to use Item_func_sp::m_handler instead,
which is always reliable.

Note, this patch is only a cleanup for MDEV-28166 to quickly fix the regression.
It fixes MDEV-28267. But it does not fix the core problem:
The code behind I_S does not take into account that the SP
cache can be updated while evaluating rows of the COLUMNS table.
This is a corner case and it never happens with any other tables.
I_S.COLUMNS is very special.

Another example of the core problem is reported in MDEV-25243.
The code accesses to Item_sp::m_sp->m_chistics of an
already freed m_sp, again. It will be addressed separately.
2022-04-09 23:01:26 +04:00
Sergei Golubchik
d623b5a1dd MDEV-22282 When using mysqldump to backup a view that contains derived tables, the database name is prepended to each table in the view
derived tables have db = "", table_name = "*", those aren't real names
to be compared with.
2022-04-09 11:49:11 +02:00
Sergei Golubchik
cfdb621243 MDEV-28255 "Error" instead of NULL in P_S.THREADS_CONNECTION_TYPE for background threads
use vio_type_names[] values as in MySQL
2022-04-09 10:46:10 +02:00
Marko Mäkelä
d90c5ddd8b MDEV-27234 fixup: Add a result file
Noticed by Monty
2022-04-09 11:12:34 +03:00
Nayuta Yanagisawa
d8463b64b3 MDEV-27239 Spider: Assertion `thd->transaction->stmt.ha_list == __null || trans == &thd->transaction->stmt' failed in ha_commit_trans on BEGIN WORK after FTWRL
The check on the SQL command type, in ha_spider::external_lock() is deleted
by e954d9de. This resulted in the wrong call of spider_internal_start_trx()
(and thus Ha_trx_info::register_ha()).

I reverted the check and refactored ha_spider::external_lock().
2022-04-08 17:59:50 +09:00
Nayuta Yanagisawa
27b5d814e2 MDEV-27065 Partitioning tables with custom data directories moves data back to default directory
The partitioning engine does not support the table-level DATA/INDEX
DIRECTORY specification.

If one create a non-partitioned table with the DATA/INDEX DIRECTORY
option and then performs ALTER TABLE ... PARTITION BY on it, the
DATA/INDEX DIRECTORY specification of the old schema is ignored.

The behavior might be a bit surprising for users because the value
of a usual table option applies to all the partitions. Thus, we raise
a warning on such ALTER TABLE ... PARTITION BY.
2022-04-08 16:49:10 +09:00
Alexander Barkov
3814b04d6b MDEV-28062 Assertion `(length % 4) == 0' failed in my_lengthsp_utf32 on INSERT..SELECT
Adding an MTR test only.

This problem was earlier fixed by the patch for:
  MDEV-28078 Garbage on multiple equal ENUMs with tricky character sets
2022-04-08 11:36:31 +04:00
Norio Akagi
161fd2d29c
MDEV-28226 Spider: remove #ifdef HANDLER_HAS_NEED_INFO_FOR_AUTO_INC 2022-04-08 15:49:19 +09:00
Nayuta Yanagisawa
4194f7b605 MDEV-25116 Spider: IF(COUNT( trigger SQL Error (1054)_ Unknown column '' in field list
The original query "SELECT IF(COUNT(a.`id`)>=0,'Y','N') FROM t" is
transformed to "SELECT COUNT(a.`id`), IF(ref >= 0, 'Y', 'N') FROM t",
where ref is Item_ref to "COUNT(a.`id`)", by split_sum_func().

Spider walks the item list twice, invoking spider_db_print_item_type().
The first invocation is in spider_create_group_by_handler() with
str == NULL. The second one is in spider_group_by_handler::init_scan()
with str != NULL.

spider_db_print_item_type() prints nothing at the first invocation,
and it prints item at the second invocation. However, at the second
invocation, the above mentioned ref to "COUNT(a.`id`)" points to
a field in a temporary table where the result will be stored. Thus,
to look behind the item_ref, Spider need to generate the query earlier.

A possible fix would be to generate a query to send in
spider_create_group_by_handler(). However, the fix requires a
considerable amount of changes of the Spider's GROUP BY handler.
I'd like to avoid that.

So, I fix the problem by not to use the GROUP BY handler when a
query contains Item_ref whose table_name, name, and alias_name_used
are not set.
2022-04-08 15:27:33 +09:00
Sergei Golubchik
b725a91757 MDEV-28253 Mysqldump - INVISIBLE column error 2022-04-07 23:02:23 +02:00
Oleg Smirnov
b2ecb622a6 Remove a garbage file from mysql-test 2022-04-07 21:09:43 +07:00
Sergei Petrunia
5a8766a980 Better comments in Item_in_subselect::inject_in_to_exists_cond() 2022-04-07 15:40:34 +03:00
Julius Goryavsky
7a03128faf MDEV-28205: SST via mariabackup stops on failure while archiving logs
Currenly SST script for mariabackup stops on any failure while archiving
logs, e.g. when unable to create directory, insufficient permissions, gzip
failure, etc. However, in case of such problems, the script should issue
a warning and continue without archiving, but not exit with a fatal error.

This commit adds this fix to the SST script for mariabackup.
2022-04-07 13:26:36 +02:00
Marko Mäkelä
6645b1d2c7 Merge 10.4 into 10.5 2022-04-07 10:36:58 +03:00
Marko Mäkelä
7b957316cb Merge 10.3 into 10.4 2022-04-07 10:32:56 +03:00
Daniel Black
dea4e178fe deb: make --output-sync=target
Rather than Debian logs containing a barely decipherable mix
of build command and the output of some other command, we
use the make option --output-sync=target. This make the compile
line and the output stay together in the output stream.

This option exists even in the make version in debian;stretch
so should work everywhere.

Test on debian:stretch, ubuntu:18.04, the two lowest version that
use the debian/rules.
2022-04-07 16:05:02 +10:00
Daniel Black
8990ffe62a MDEV-28153: Debian autobake to generate control
Without doing the full build.

Autobake now includes a dependency on lsb-release.

As the BB CI images
(https://github.com/MariaDB/mariadb.org-tools/blob/master/buildbot.mariadb.org/ci_build_images/debian.Dockerfile)
have explicit dependencies, there's no point maintaining them in
two places. We don't want do the full autobake-deb.sh there, just enough
to have the control file containing the correct dependencies.

Helps: https://github.com/MariaDB/mariadb.org-tools/pull/130
2022-04-07 16:05:02 +10:00
Jan Lindström
3c99a48db3 MDEV-28247 : Disable background ibuf merge during Galera SST
This failure was caused by MDEV-25975, which removed the parameter
innodb_disallow_writes.

Added a check for wsrep_sst_disable_writes to the function
ibuf_merge_in_background().
2022-04-07 08:45:01 +03:00
Daniel Black
4ee00a29e3 MDEV-28250 aix test case failure innodb_zip.innochecksum_3,4k,crc32,innodb
As discovered by tracing, but also presenting in AIX fseeko
documentation, seeking beyond the EOF is acceptable, as you can write
there.

To display the same error in AIX to other implementations that return
errors on seek, we take the EFBIG error code on reading and error the
same way.

An AIX truss of an aspect of the test:

truss extra/innochecksum --page=18446744073709551615 $MYSQLD_DATADIR/test/tab1.ibd

  statx("./mysql-test/var/log/innodb_zip.innochecksum_3-4k,crc32,innodb/mysqld.1/data//test/tab1.ibd", 0x0FFFFFFFFFFFF610, 176, 010) = 0
  kopen("./mysql-test/var/log/innodb_zip.innochecksum_3-4k,crc32,innodb/mysqld.1/data//test/tab1.ibd", O_RDONLY|O_LARGEFILE) = 3
  kfcntl(3, 12, 0x00000001100006C8)               = 0
  kfcntl(3, F_GETFL, 0x00000001100A6CF8)          = 67108864
  kioctl(3, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
  klseek(3, 0, 1, 0x0FFFFFFFFFFFF3F0)             = 0
  kioctl(3, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
  kread(3, "DEADBEEF\0\0\0\0FFFFFFFF".., 4096)    = 4096
  klseek(3, 0, 1, 0x0FFFFFFFFFFFF450)             = 0
  klseek(3, 17592186040320, 0, 0x0FFFFFFFFFFFF450) = 0
  klseek(3, 0, 1, 0x0FFFFFFFFFFFF3F0)             = 0
  kread(3, "DEADBEEF\0\0\0\0FFFFFFFF".., 4096)    Err#27 EFBIG

An equivalent Linux trace:

ltrace extra/innochecksum --page=18446744073709551615 $MYSQLD_DATADIR/test/tab1.ibd

  stat64(0x7fff10ea2dc3, 0x7fff10ea0670, 88, 0x8026be41)         = 0
  open64("./mysql-test/var/log/innodb_zip."..., 0, 02072403160)  = 3
  fcntl64(3, 6, 0x139f180, 1)                                    = 0
  fgetpos64(0x615000000080, 0x7fff10ea0760, 1, 0)                = 0
  fseeko64(0x615000000080, 0xffffffff000, 0, 5 <unfinished ...>
  pthread_getspecific(0, 0x4d0eb8, 0x7fff10ea0490, 0)            = 0x7f7b2806d000
  <... fseeko64 resumed> )                                       = 0
  fgetpos64(0x615000000080, 0x7fff10ea0760, 1, 1)                = 0
  feof(0x615000000080)                                           = 0
  feof(0x615000000080)                                           = 1
  Error: Unable to seek to necessary offset
2022-04-07 15:17:52 +10:00
Daniel Black
e84e134a91 main.thread_pool_info - no threadpool on aix 2022-04-07 15:17:52 +10:00
Thirunarayanan Balathandayuthapani
27b0030b9d MDEV-27783 InnoDB: Failing assertion: table->get_ref_count() == 0 upon ALTER TABLE ... MODIFY COLUMN
- There is a race condition occurs between purge thread and DDL.
So purge thread can increment n_ref_count even after DDL does
purge_sys_t::stop_FTS().

- dict_table_open_on_id for purge thread should check
purge_sys.must_wait_FTS() before acquring the table.

- purge_sys.stop_FTS() does acquire dict_sys.latch for setting
the purge system flag and check table ref count on auxilary tables.
2022-04-07 10:32:51 +05:30
Alexander Barkov
7355f7b1f5 Adding MTR tests to cover how keywords of different kinds behave in various contexts 2022-04-07 06:13:22 +04:00
Daniel Black
e2e4aa29db deb: test restrictions missed in merge
Affects Salsa autopkgtest.
2022-04-07 09:17:14 +10:00
Oleg Smirnov
53b580a91c MDEV-28077 'Wrong create options' error with 'big_tables' enabled
The cause of the bug is overflow of uint16 KEY_PART_INFO::length and/or
uint16 KEY_PART_INFO::store_length. The solution is to increase the size
of those variables to the 'uint' type (which is 32-bit long)
2022-04-06 15:28:34 +03:00
Oleg Smirnov
85192553ae MDEV-24560 SIGSEGV in st_join_table::cleanup
If JOIN::create_postjoin_aggr_table encounters errors during execution
then free_tmp_table() is then called twice for JOIN_TAB::aggr.
The solution is to initialize JOIN_TAB::aggr only on successful completion
of JOIN::create_postjoin_aggr_table
2022-04-06 15:12:07 +03:00
Marko Mäkelä
6cb6ba8b7b Merge 10.8 into 10.9 2022-04-06 13:33:33 +03:00
Marko Mäkelä
b2baeba415 Merge 10.7 into 10.8 2022-04-06 13:28:25 +03:00