Commit graph

194869 commits

Author SHA1 Message Date
Vladislav Vaintroub
012e724deb MDEV-27796 Windows - starting server with huge innodb-log-buffer-size may fail
Fixed tpool::pread() and tpool::pwrite() to return SSIZE_T on Windows,
so that huge numbers are not converted to negatives.

Also, make sure to never attempt reading/writing more bytes than
DWORD can accomodate (4G)
2022-02-10 17:25:12 +01:00
Sergei Golubchik
9e2c26b0f6 MDEV-26351 segfault - (MARIA_HA *) 0x0 in ha_maria::extra
don't let Aria create a table that it cannot open
2022-02-10 15:48:06 +01:00
Sergei Golubchik
1b8bb44106 MDEV-26351 segfault - (MARIA_HA *) 0x0 in ha_maria::extra
use the correct check. before invoking handler methods we
need to know that the table was opened, not only created.
2022-02-10 15:48:06 +01:00
Oleksandr Byelkin
0168d1eda3 MDEV-25766 Unused CTE lead to a crash in find_field_in_tables/find_order_in_list
Do not assume that subquery Item always present.
2022-02-10 14:36:25 +01:00
Sergei Golubchik
9e39d0ae44 MDEV-25787 Bug report: crash on SELECT DISTINCT thousands_blob_fields
fix a debug assert to account for not opened temp tables
2022-02-10 13:45:11 +01:00
Monty
ad1fb06982 MDEV-27789 mysql_upgrade / mariadb-upgrade in 10.6.6 is putting password in host argument
Removed all dependencies of command line arguments based on positions in
an array (this kind of code should never have been written).
Instead use option names, which are stable.

Reviewer: Sergei Golubchik
2022-02-10 12:29:37 +02:00
Marko Mäkelä
cce994057b Merge 10.5 into 10.6 2022-02-09 15:49:50 +02:00
Marko Mäkelä
fd101daa84 MDEV-27716 mtr_t::commit() acquires log_sys.mutex when writing no log
mtr_t::is_block_dirtied(), mtr_t::memo_push(): Never set m_made_dirty
for pages of the temporary tablespace. Ever since
commit 5eb539555b
we never add those pages to buf_pool.flush_list.

mtr_t::commit(): Implement part of mtr_t::prepare_write() here,
and avoid acquiring log_sys.mutex if no log is written.
During IMPORT TABLESPACE fixup, we do not write log, but we must
add pages to buf_pool.flush_list and for that, be prepared
to acquire log_sys.flush_order_mutex.

mtr_t::do_write(): Replaces mtr_t::prepare_write().
2022-02-09 15:10:10 +02:00
Oleksandr Byelkin
1bed56400e Merge branch '10.6' into bb-10.6-release 2022-02-09 09:05:27 +01:00
Oleksandr Byelkin
34c5019698 Merge branch '10.5' into bb-10.5-release 2022-02-09 08:57:41 +01:00
Oleksandr Byelkin
8a7776a83d Merge branch '10.4' into bb-10.4-release 2022-02-09 08:53:02 +01:00
Oleksandr Byelkin
e352444558 Merge branch '10.3' into bb-10.3-release 2022-02-09 08:49:00 +01:00
Oleksandr Byelkin
941bc70536 Merge branch '10.2' into bb-10.2-release 2022-02-09 08:44:48 +01:00
Marko Mäkelä
5c46751f23 MDEV-27734 Set innodb_change_buffering=none by default
The aim of the InnoDB change buffer is to avoid delays when a leaf page
of a secondary index is not present in the buffer pool, and a record needs
to be inserted, delete-marked, or purged. Instead of reading the page into
the buffer pool for making such a modification, we may insert a record to
the change buffer (a special index tree in the InnoDB system tablespace).
The buffered changes are guaranteed to be merged if the index page
actually needs to be read later.

The change buffer could be useful when the database is stored on a
rotational medium (hard disk) where random seeks are slower than
sequential reads or writes.

Obviously, the change buffer will cause write amplification, due to
potentially large amount of metadata that is being written to the
change buffer. We will have to write redo log records for modifying
the change buffer tree as well as the user tablespace. Furthermore,
in the user tablespace, we must maintain a change buffer bitmap page
that uses 2 bits for estimating the amount of free space in pages,
and 1 bit to specify whether buffered changes exist. This bitmap needs
to be updated on every operation, which could reduce performance.

Even if the change buffer were free of bugs such as MDEV-24449
(potentially causing the corruption of any page in the system tablespace)
or MDEV-26977 (corruption of secondary indexes due to a currently
unknown reason), it will make diagnosis of other data corruption harder.

Because of all this, it is best to disable the change buffer by default.
2022-02-09 08:36:41 +02:00
Daniel Bartholomew
fa73117bf8
bump the VERSION 2022-02-08 17:51:29 -05:00
Daniel Bartholomew
f7704d74cb
bump the VERSION 2022-02-08 17:31:40 -05:00
Daniel Bartholomew
2f07b21cf3
bump the VERSION 2022-02-08 17:10:48 -05:00
Daniel Bartholomew
30cc63fa1a
bump the VERSION 2022-02-08 16:47:37 -05:00
Daniel Bartholomew
c0a44ff7f1
bump the VERSION 2022-02-08 16:28:44 -05:00
Monty
38058c04a4 MDEV-26585 Wrong query results when using index for group-by
The problem was that "group_min_max optimization" does not work if
some aggregate functions, like COUNT(*), is used.
The function get_best_group_min_max() is using the join->sum_funcs
array to check which aggregate functions are used.
The bug was that aggregates in HAVING where not yet added to
join->sum_funcs at the time get_best_group_min_max() was called.

Fixed by populate join->sum_funcs already in prepare, which means that
all sum functions will be in join->sum_funcs in get_best_group_min_max().
A benefit of this approach is that we can remove several calls to
make_sum_func_list() from the code and simplify the function.

I removed some wrong setting of 'sort_and_group'.
This variable is set when alloc_group_fields() is called, as part
of allocating the cache needed by end_send_group() and does not need
to be set by other functions.

One problematic thing was that Spider is using *join->sum_funcs to detect
at which stage the optimizer is and do internal calculations of aggregate
functions. Updating join->sum_funcs early caused Spider to fail when trying
to find min/max values in opt_sum_query().
Fixed by temporarily resetting sum_funcs during opt_sum_query().

Reviewer: Sergei Petrunia
2022-02-08 14:32:29 +02:00
Monty
d314bd2664 MDEV-27442 Wrong result upon query with DISTINCT and EXISTS subquery
The problem was that get_best_group_min_max() did not check if fields used
by the "group_min_max optimization" where used in sub queries.
Because of this, it did not detect that a key (b,a) was used in the WHERE
clause for the statement:
SELECT DISTINCT b FROM t1 WHERE EXISTS ( SELECT 1 FROM DUAL WHERE a > 1 ).

Fixed by also traversing the sub queries when checking if a field is used.
This disables group_min_max_optimization for the above query.

Reviewer: Sergei Petrunia
2022-02-08 14:32:28 +02:00
Monty
a1c2380753 MENT-328 Retry BACKUP STAGE BLOCK DDL in case of deadlocks
MENT-328 wrongly assumed that the backup failed because of warnings from
mariabackup about not found files. This is normal (and the error message
should be deleted).

randgen failed because mariabackup didn't retry BACKUP STAGE BLOCK DDL
if it failed with a deadlock.

To simplify things, I implemented the retry loop in the server as
this particular deadlock should be quickly resolved.
2022-02-08 14:32:28 +02:00
Monty
0ec27d7b1f Don't run innodb_defgragment under valgrind (too slow) 2022-02-08 14:32:28 +02:00
Monty
88fb89acb7 Fixes some compiler issues on AIX ( 2022-02-08 14:32:28 +02:00
Monty
df02de68f3 Fixed my_addr_resolve (cherry picked from 10.6)
When a server is compiled with -fPIE, my_addr_resolve needs to
subtract the info.dli_fbase from symbol addresses in memory for
addr2line to recognize them.  When a server is compiled without -fPIE,
my_addr_resolve should not do it.  Unfortunately not all compilers
define __PIE__ when -fPIE was used (e.g. older gcc doesn't), so we
have to resort to run-time detection.
2022-02-08 14:32:28 +02:00
Vladislav Vaintroub
ae33a006f7 MDEV-27738 Windows : mysql-test-run --extern does not work
Use portable quoting in mtr_add_arg.
2022-02-08 10:14:50 +01:00
Teemu Ollakka
21413aee0a MDEV-27737 Wsrep SST scripts not working on FreeBSD
- Changed SST scripts to use /usr/bin/env bash instead of
  /bin/bash for better portability.
- Fixed use of mktemp on non-Linux platforms to produce
  temporary file instead of directory.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2022-02-08 08:02:01 +02:00
Sergei Petrunia
5c89386fdb MDEV-17785: Window functions not working in ONLY_FULL_GROUP_BY mode
(Backport Varun Gupta's patch + edit the commit comment)

Name resolution code produced errors for valid queries with window
functions (but not for queries which used aggregate functions as
window functions).

Name resolution code worked incorrectly, because window function
objects had is_window_func_sum_expr()=false. This was so, because
mark_as_window_func_sum_expr() was only called for aggregate functions
used as window functions.

The fix is to call it for any window function.
2022-02-07 21:43:42 +03:00
Vladislav Vaintroub
881918bf77 MDEV-27754 : Assertion with innodb_flush_method=O_DSYNC
If innodb_flush_method=O_DSYNC, log_sys.flushed_to_disk_lsn  is changed
without 'flush_lock' protection inside log_write().

This leads to a race condition, if there are 2 threads running in parallel,
doing log_write_up_to() with different values for 'flush_to_disk'

In this case, log_write() and log_write_flush_to_disk_low() can execute at
the same time, and both would change flushed_lsn.

The fix is to remove special treatment of durable writes from log_write().
There is no apparent reason for this special treatment, log_write_flush_to_disk_low()
is already optimized for durable writes.

Nor there is an apparent reason to call log_flush_notify() more often in
for O_DSYNC.
2022-02-07 09:14:00 +01:00
Sergei Golubchik
a319220e62 update test result 2022-02-06 23:00:34 +01:00
Sergei Golubchik
4ffffd98a5 update columnstore 2022-02-05 14:50:25 +01:00
Oleksandr Byelkin
a806c993e7 Fix for compiling under clang. 2022-02-04 14:40:42 +01:00
Oleksandr Byelkin
d87979b48c Merge branch '10.5' into 10.6 2022-02-04 10:01:08 +01:00
Oleksandr Byelkin
ad3ac55641 fix 32bit embedded result file. 2022-02-04 09:55:04 +01:00
Oleksandr Byelkin
2cf52736de Fix for clang compilation 2022-02-04 09:54:45 +01:00
Marko Mäkelä
82f5981e72 MDEV-27058 fixup: Crash in innodb.leaf_page_corrupted_during_recovery
buf_page_get_low(): If the page was read-fixed, validate the page ID
because the page could have been marked as corrupted. We should retry
the page read in this case, instead of returning a soon-to-be-evicted
corrupted page to the caller.

This was initially only observed on Microsoft Windows.
On Linux, this was repeated after adding a sleep
to buf_pool_t::corrupted_evict() between
bpage->zip.fix.fetch_sub() and bpage->lock.x_unlock().
2022-02-03 17:02:27 +01:00
Marko Mäkelä
05c33d6216 MDEV-27736 Allow seamless upgrade despite ROW_FORMAT=COMPRESSED
In commit 9bc874a594 (MDEV-23497)
the configuration option innodb_read_only_compressed was introduced
to giver users advance notice of a plan to remove ROW_FORMAT=COMPRESSED
support for InnoDB.

Based on user feedback, this plan has been scrapped.
Even though ROW_FORMAT=COMPRESSED is a dead end and causes some
overhead for InnoDB data structures, we can live with that.

Now that we know that some users really want to keep using
ROW_FORMAT=COMPRESSED, the previous default value of the parameter
innodb_read_only_compressed=ON should be changed to OFF, to allow
smooth upgrades to 10.6 and later versions, without requiring users
to update any configuration file.
2022-02-03 17:02:14 +01:00
Oleksandr Byelkin
f5c5f8e41e Merge branch '10.5' into 10.6 2022-02-03 17:01:31 +01:00
Andrei
e53199e76b MDEV-27721 rpl.rpl_relay_max_extension test is not FreeBSD-compatible
Fixed the test to execute only on linux as it depends on unportable sed.
2022-02-02 20:02:20 +02:00
Oleksandr Byelkin
cf63eecef4 Merge branch '10.4' into 10.5 2022-02-01 20:33:04 +01:00
Thirunarayanan Balathandayuthapani
8d742fe4ac MDEV-26326 mariabackup skip valid ibd file
- Store the deferred tablespace name while loading the tablespace
for backup process.

- Mariabackup stores the list of space ids which has page0 INIT_PAGE
records. backup_first_page_op() and first_page_init() was introduced
to track the page0 INIT_PAGE records.

- backup_file_op() and log_file_op() was changed to handle
FILE_MODIFY redo log records. It is used to identify the
deferred tablespace space id.

- Whenever file operation redo log was processed by backup,
backup_file_op() should check whether the space name exist
in deferred tablespace. If it is then it needs to store the
space id, name when FILE_MODIFY, FILE_RENAME redo log processed
and it should delete the tablespace name from defer list in other
cases.

- backup_fix_ddl() should check whether deferred tablespace has
any page0 init records. If it is then consider the tablespace
as newly created tablespace. If not then backup should try
to reload the tablespace with SRV_BACKUP_NO_DEFER mode to
avoid the deferring of tablespace.
2022-02-01 19:50:08 +05:30
Vladislav Vaintroub
62ba2f230a MDEV-27535 remove debug output 2022-01-31 21:50:57 +01:00
Vladislav Vaintroub
8c457dad48 MDEV-27546 MSI should fail if INSTALLDIR is a non-empty existing directory
Make installer fail if INSTALLDIR exists.
In UI mode, show a popup and disable "Next" button until install root is
corrected
2022-01-31 20:10:08 +01:00
Vladislav Vaintroub
12cea09713 MDEV-27535 Service does not start after MSI install into restricted directory
This happens for example if one installs into home directory of a user
C:\Users\<username>\mariadb

The reason is that the service user "NT SERVICE\<service_name>" does
not have read and execute permissions for service executable mysqld.exe
in this directory.
Moreover, it would not have read permissions for server.dll loaded by the
exe, or to plugin directory, where plugins may reside.

The fix is to give service users read and execute permissions to bin, share, and
lib\plugin subdirectories.

The permission setting is doneby  mysql_install_db.exe, but also in MSI.
It is important to do that in MSI, as we want permissions to survive
the MSI upgrade.
2022-01-31 20:10:08 +01:00
Alexander Barkov
2a0962f39b MDEV-27696 Json table columns accept redundant COLLATE syntax 2022-01-31 21:18:43 +04:00
Sergei Golubchik
fb40a2fabf pass MYSQL_MAINTAINER_MODE down to srpm builds
fixes errors on rpm-*-debug builder
2022-01-31 11:45:30 +01:00
Sergei Golubchik
0943386fb3 fix main.mysqld--help-aria failures
when it's run directly after main.mysql_json_mysql_upgrade

because mysqld--help-aria starts a second mysqld that reads the plugin
table, so it has to be flushed and closed at that time.
2022-01-31 11:41:12 +01:00
Oleksandr Byelkin
c04a203a10 Rocksdb result fix after merge 2022-01-31 08:37:33 +01:00
Sergei Golubchik
2d4a0f6079 pass MYSQL_MAINTAINER_MODE down to srpm builds
fixes errors on rpm-*-debug builder
2022-01-30 22:52:59 +01:00
Oleksandr Byelkin
2d85188627 Fix galera result after merge 2022-01-30 18:14:27 +01:00