Commit graph

4651 commits

Author SHA1 Message Date
Anthony Ryan
babd833685 MDEV-29914: Fix maridab-upgrade when sql_safe_updates = on is set in my.cnf
Tested multiple major version upgrades with sql_safe_updates enabled, and
confirmed the issue is resolved.

Reviewer: Daniel Black
2023-10-23 19:25:10 +11:00
hotairballoon-3573
19eac149b1 MDEV-32142 mariadb-install-db shows warning on missing directory /auth_pam_tool_dir
Without pam compiled there will be no auth_pam_tool_dir, so check this
before attempting something that will error.

Reviewer: Sergei Golubchik / Daniel Black
2023-10-18 21:14:48 +11:00
Julius Goryavsky
073a088f31 MDEV-31467: wsrep_sst_mariabackup not working on FreeBSD
Due to the different command line format of the timeout
utility on FreeBSD and Linux, SST scripts for mariabackup
may not work on FreeBSD. This commit fixes this problem
by adding a different command to test options on FreeBSD
and adding proper formatting for the utility options.
2023-10-17 13:00:13 +02:00
Julius Goryavsky
8660e2de0e MDEV-29893: SST fails when having datadir set to a symlink
SST for mariabackup may not destroy old files if datadir or
other working directory is declared as a symlink due to the lack
of the "-L" option among the find utility options, similarly SST
for rsync in some cases may not transfer data directories if they
are created as symlinks. This fix adds the missing option and
generally unifies the work with find utility options to avoid
failures in the interpretation of directories and regular
expressions.
2023-10-13 02:13:26 +02:00
Daniel Black
9b5275b8f5 MDEV-31332: Galera rsync sst to ignore .snapshot/ files
.snapshot exists as a directory on NetApp storage and
should not be copied during the sst process.

Thanks Daniel Czadek for the bug report.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-09-26 08:42:39 +02:00
Oleksandr Byelkin
2bf291ba59 MDEV-30820 slow log Rows_examined out of range
Fix row counters to be able to get any possible value.
2023-09-22 12:10:38 +02:00
Daniel Black
bf5c251239 MDEV-27757 Database upgrade fails from 5.1: slow_log table
The table structure from MySQL-5.1.14 is:

CREATE TABLE `slow_log` (
  `start_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `user_host` mediumtext NOT NULL,
  `query_time` time NOT NULL,
  `lock_time` time NOT NULL,
  `rows_sent` int(11) NOT NULL,
  `rows_examined` int(11) NOT NULL,
  `db` varchar(512) DEFAULT NULL,
  `last_insert_id` int(11) DEFAULT NULL,
  `insert_id` int(11) DEFAULT NULL,
  `server_id` int(11) DEFAULT NULL,
  `sql_text` mediumtext NOT NULL
) ENGINE=CSV DEFAULT CHARSET=utf8 COMMENT='Slow log'

Even as far back as MySQL-5.5.40 this table could be created as NULLs
where not permitted in the CSV table time, but it seems they
where allowed sometime.

As the first part of mariadb-upgrade adds the column thread_id without
correcting the 'NULL'able status of existing columns it fails.

We reorder the sql statements in the ugprade as follows:

ALTER TABLE slow_log MODIFY {columns} {new types} NOT NULL,....

As thread_id doesn't exist in the above statement it was removed from
the first ALTER TABLE statement to prevent failure.

Previous ALTER TABLE slow_log where moved later appending thread_id
and rows_affected, and also enforces the type of thread_id if it
was incorrectly like the now first ALTER STATEMENT slow_log used
to do.
2023-09-20 08:52:09 +10:00
Daniel Black
8bbe3a3cd2 MDEV-21194: mariadb-install-db doesn't properly grant proxy privileges to all created user accounts
mariadb-install-db --auth-root-authentication-method=normal created 4
root accounts by default, but only two of these had PROXY privilege
granted.

mariadb-install-db (default option
--auth-root-authentication-method=socket) as non-root user also didn't
grant PROXY priv to the created nonroot@localhost user.

To fix this, in mysql_system_tables_data.sql, we re-use tmp_user_nopasswd
as this contains the list of all root users.

REPLACE INTO tmp_proxies_priv SELECT @current_hostname, IFNULL(@auth_root_socket, 'root')
creates the $user@$current_host but will not error if @auth_root_socket
is null. Note @current_hostname lines are filtered out with
--cross-bootstrap in mariadb-install-db so it was needed to include this
expression for consistency.

Like the existing mysql_system_tables.sql is used to create teh
$user@localhost proxies_priv.

Test cases roles.acl_statistics, perfschema,privilege_table_io depends on the number of proxy users.

After:

--auth-root-authentication-method=normal:

MariaDB [mysql]> select * from global_priv;
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
| Host      | User        | Priv                                                                                                                     |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
| localhost | mariadb.sys | {"access":0,"plugin":"mysql_native_password","authentication_string":"","account_locked":true,"password_last_changed":0} |
| localhost | root        | {"access":18446744073709551615}                                                                                          |
| bark      | root        | {"access":18446744073709551615}                                                                                          |
| 127.0.0.1 | root        | {"access":18446744073709551615}                                                                                          |
| ::1       | root        | {"access":18446744073709551615}                                                                                          |
| localhost |             | {}                                                                                                                       |
| bark      |             | {}                                                                                                                       |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
7 rows in set (0.001 sec)

MariaDB [mysql]> select * from proxies_priv;
+-----------+------+--------------+--------------+------------+---------+---------------------+
| Host      | User | Proxied_host | Proxied_user | With_grant | Grantor | Timestamp           |
+-----------+------+--------------+--------------+------------+---------+---------------------+
| localhost | root |              |              |          1 |         | 2023-07-10 12:12:24 |
| 127.0.0.1 | root |              |              |          1 |         | 2023-07-10 12:12:24 |
| ::1       | root |              |              |          1 |         | 2023-07-10 12:12:24 |
| bark      | root |              |              |          1 |         | 2023-07-10 12:12:24 |
+-----------+------+--------------+--------------+------------+---------+---------------------+

--auth-root-authentication-method=socket:

MariaDB [mysql]> select * from proxies_priv;
+-----------+------+--------------+--------------+------------+---------+---------------------+
| Host      | User | Proxied_host | Proxied_user | With_grant | Grantor | Timestamp           |
+-----------+------+--------------+--------------+------------+---------+---------------------+
| localhost | root |              |              |          1 |         | 2023-07-10 12:11:55 |
| localhost | dan  |              |              |          1 |         | 2023-07-10 12:11:55 |
| bark      | dan  |              |              |          1 |         | 2023-07-10 12:11:55 |
+-----------+------+--------------+--------------+------------+---------+---------------------+
3 rows in set (0.017 sec)

MariaDB [mysql]> select * from global_priv;
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Host      | User        | Priv                                                                                                                                       |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| localhost | mariadb.sys | {"access":0,"plugin":"mysql_native_password","authentication_string":"","account_locked":true,"password_last_changed":0}                   |
| localhost | root        | {"access":18446744073709551615,"plugin":"mysql_native_password","authentication_string":"invalid","auth_or":[{},{"plugin":"unix_socket"}]} |
| localhost | dan         | {"access":18446744073709551615,"plugin":"mysql_native_password","authentication_string":"invalid","auth_or":[{},{"plugin":"unix_socket"}]} |
| localhost |             | {}                                                                                                                                         |
| bark      |             | {}                                                                                                                                         |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.000 sec)

MariaDB [mysql]> show grants;
+----------------------------------------------------------------------------------------------------------------------------------------+
| Grants for dan@localhost                                                                                                               |
+----------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO `dan`@`localhost` IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket WITH GRANT OPTION |
| GRANT PROXY ON ''@'%' TO 'dan'@'localhost' WITH GRANT OPTION                                                                           |
+----------------------------------------------------------------------------------------------------------------------------------------+
2023-09-19 08:07:07 +10:00
Andrew Hutchings
48e6918c94 Revert "update galera_new_cluster to use environment file"
This reverts commit b54e4bf00b.
2023-08-08 15:46:39 +01:00
Christian Hesse
b54e4bf00b update galera_new_cluster to use environment file
Now that the systemd unit files use an environment file to pass
_WSREP_START_POSITION we have to update galera_new_cluster as well.
2023-08-02 17:16:37 +01:00
Julius Goryavsky
d1a4315f4c MDEV-30402: Encrypted mariabackup SST breaks on distributions with newer socat
This commit adds a new 'no-sni' option to socat which is required to
properly authenticate with newer socat versions (after version 1.7.4+).
This option is needed to disable the automatic use of the SNI feature
(Server Name Indication) since the SST script directly specifies the
commonname if necessary and automatic activation of the SNI feature
is unnecessary in such scenarios.
2023-04-13 07:49:35 +02:00
Lorna Luo
acfb5dfd97 MDEV-22683: Ensure system tables are correctly upgraded in MariaDB 10.4
Running mysql_upgrade should end up with the exact same system tables as fresh
installations have after running mysql_install_db. To ensure the upgrade is
correct and complete:

- Remove the redundant modification of thread_id`. On 5.5 version, the
  `general_log` table was created as `CREATE TABLE IF NOT EXISTS general_log
  (..., thread_id INTEGER NOT NULL, ...)`, and starting from 10.0+, the table is
  created as `CREATE TABLE IF NOT EXISTS general_log (..., thread_id BIGINT(21)
  UNSIGNED NOT NULL, ...)`, but mysql_upgrade is not properly upgrading the
  table. It modifies the `thread_id` twice in one query, which could leave the
  table not modified and lead to other potential error when upgrading from
  MariaDB 5.5 or older.

- Update `servers` to ensure `Host` and `User` has correct data type if
  upgrading from 10.1 or older. On versions 10.0 and 10.1, the `servers` table
  was created as `CREATE TABLE IF NOT EXISTS servers (..., Host char(64) NOT
  NULL DEFAULT , ..., Owner char(64) NOT NULL DEFAULT , ...)`, and starting
  from 10.2, the table is created as `CREATE TABLE IF NOT EXISTS servers (...,
  Host varchar(2048) NOT NULL DEFAULT , ..., Owner varchar(512) NOT NULL
  DEFAULT , ...)`.

All new code of the whole pull request, including one or several files that
are either new files or modified ones, are contributed under the BSD-new license.
I am contributing on behalf of my employer Amazon Web Services, Inc.
2023-03-02 09:06:35 +11:00
Ian Gilfillan
801c0b4b47 Update 10.4 HELP tables 2023-01-24 09:52:11 +11:00
Marko Mäkelä
71e8e4934d Merge 10.3 into 10.4 2023-01-13 09:28:25 +02:00
lilinjie
eb145e5ad7 fix typos
Signed-off-by: lilinjie <lilinjie@uniontech.com>
2023-01-12 14:02:20 +11:00
Julius Goryavsky
53c4be7bc0 MDEV-30220: rsync SST completely ignores aria-log-dir-path
This commit adds support for the --aria-log-dir-path
option on the command line and for the aria-log-dir-path
option in the configuration file to the SST scripts, since
before this change these parameters were completely ignored
during SST - SST scripts assumed that aria logs files are
always located in the same directory as logs for innodb.

Tests for this change will be added as a separate commit,
along with tests for MDEV-30157 and MDEV-28669.
2023-01-11 15:01:00 +01:00
Julius Goryavsky
b84f3fa769 MDEV-30157: Galera SST doesn't properly handle undo* files from innodb
This fix adds separate handling for "undo*" files that contain undo
logs as part of innodb files and adds a filter for undo* to the main
filter used when initially transferring files with rsync.
2023-01-11 15:01:00 +01:00
Julius Goryavsky
e4a4aad7cf pre-MDEV-30157 & pre-MDEV-28669: fixes before the main corrections
This commit adds even more correct handling of parameters
with paths when they contain leading or trailing spaces and/or
slashes. Also it fixes problems that occur when the user specified
explicit paths to additional directories, but these paths match
the specified path of the data directory - in this case, additional
subdirectories should be treated (in relation to the data directory)
in the same way as if these paths were not specified or as if they
are implicitly specified as "." or "./". But prior to this fix,
existing code treated any values as if they were completely
separate directories, whether or not they actually point to the
same location to which datadir points to - and this sometimes
resulted in incorrect file transfers.

This fix does not contain separate tests, as tests will be
part of the main commit(s). This fix has been made as a separate
commit to facilitate review for major substantive fixes related
to MDEV-30157 and MDEV-28669.
2023-01-11 15:01:00 +01:00
Sergei Golubchik
fdcfc25127 Merge branch '10.3' into 10.4 2023-01-10 21:04:17 +01:00
Andrew Hutchings
e51a1d6fc0 MDEV-30329: mariadb-service-convert resets systemd service to default User=root
If mariadb-service-convert is run and the user variable is unset then
this sets `User=` in `[Service]`, which then tries to run mariadb as
root, which in-turn fails. This only happens when mysqld_safe is missing
which is all the time now. So don't set `User=` if there is no user variable.

Reviewer: Sergei Golubchik <serg@mariadb.org> (in PR )
2023-01-03 11:45:24 +11:00
Marko Mäkelä
f97f6955bd Merge 10.3 into 10.4 2022-12-14 06:20:04 +02:00
Daniel Black
acfaa04587
MDEV-18591: mysql_install_db - pass --log-error to mysqld in install ()
Previously we parsed it out in mysql_install_db for use in the error
message, but failed to pass it to mysqld in the bootstrap.

Also match log_error as it might appear in the .cnf files.

Thanks Michal Schorm for the test case.

Reviewed by: Faustin Lammler
2022-12-13 15:17:44 +01:00
Marko Mäkelä
fdf43b5c78 Merge 10.3 into 10.4 2022-12-13 11:37:33 +02:00
Christian Gonzalez
112870d7b4 MDEV-16735 Ensure mysql_upgrade works when changing alter_algorithm
MDEV-16735 describes how mysql_upgrade fails when alter_algorithm
is set to a value different than 'DEFAULT'/'COPY'. It was marked as
fixed by 0ee0868, but the fix didn't covered the possibility of having
the global value of alter_algorithm set to something different than
'DEFAULT'/'COPY'. To ensure that the upgrade process works properly
regardless the global value of alter_altorithm, this commit force it's
value to 'DEFAULT' (note the quotes) for the mysql_upgrade session.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
2022-11-29 10:55:17 +11:00
Ian Gilfillan
f19e8559aa MDEV-28701 Update 10.4 HELP tables 2022-10-25 09:40:57 +11:00
Ian Gilfillan
34ff5ca895 MDEV-28701 Update 10.3 HELP tables 2022-10-25 09:40:38 +11:00
Ian Gilfillan
ef784c4ea2 Update 10.4 HELP contents 2022-09-19 16:57:10 +10:00
Oleksandr Byelkin
48e35b8cf6 Merge branch '10.3' into 10.4 2022-08-02 14:15:39 +02:00
Julius Goryavsky
7fb1f919d0 MDEV-28758: Mariabackup copies binary logs to backup directory
This commit restores defaults and functionality regarding binlogs
to the way it was prior to MDEV-27524. The mariabackup utility no
longer saves binlogs files as part of a backup without the --galera-info
option. However, since we use --galera-info during SST, the behavior
of mariabackup changes and, in combination with GTIDs support enabled,
mariabackup transfers one (most recent) binlog file obtained after
FLUSH BINARY LOGS. In other cases, binlogs are not transferred during
SST in mariabackup mode. As for SST in the rsync mode, it works the
same way as before MDEV-27524 - by default it transfers one last
binlog file.

The --sst-max-binlogs option for mariabackup and the sst_max_binlogs
parameter in the [sst] / server sections are no longer supported for
SST via mariabackup.
2022-08-01 15:53:14 +02:00
Marko Mäkelä
392ee571c1 Merge 10.3 into 10.4 2022-07-01 13:10:36 +03:00
Marko Mäkelä
045771c050 Fix most clang-15 -Wunused-but-set-variable
Also, refactor trx_i_s_common_fill_table() to remove dead code.

Warnings about yynerrs in Bison-generated yyparse() will remain for now.
2022-07-01 09:48:36 +03:00
Marko Mäkelä
01d757036f Merge 10.3 into 10.4 2022-06-27 10:14:37 +03:00
Julius Goryavsky
3e09c6199d MDEV-26562: galera-sst-mariabackup is failing due to missing xtrabackup_checkpoints
This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).

Also fixed broken logic of automatic generation of the Diffie-Hellman parameters
for socat version less than 1.7.3 (which defaults to 512-bit values instead of
2048-bit ones).
2022-06-21 13:45:20 +02:00
Julius Goryavsky
124326d810 MDEV-28656: Inability to roll upgrade without stopping the Galera cluster 2022-06-14 12:29:14 +02:00
Marko Mäkelä
c89e3b70a7 Merge 10.3 into 10.4 2022-06-09 11:53:46 +03:00
Sergei Golubchik
7c4efab903 typo fixed: [[space]] -> [[:space:]] 2022-06-07 18:26:05 +02:00
Marko Mäkelä
96f4b4a55b Merge 10.3 into 10.4 2022-06-02 16:34:17 +03:00
Julius Goryavsky
7d3d3838c1 MDEV-28583: post-merge fixes 2022-05-23 13:11:14 +02:00
Sergei Golubchik
23ddc3518f Merge branch '10.3' into 10.4 2022-05-18 01:25:30 +02:00
Julius Goryavsky
d388e7eb86 MDEV-28583: Galera: binlogs disappear after rsync IST
This commit sends a flag indicating the presence of the "--bypass"
option from the donor node to the joiner nodes during rsync IST,
because without such a flag it is impossible to distinguish IST
from the SST on the joiner nodes (in IST/SST scripts, because the
"--bypass" option is still not passed to scripts from server code).
Specifically, this fixes an issue with binary logs disappearing
after IST (via rsync). There are also changes to diagnostic messages
here that will make it easier to diagnose script-related problems
in the future when debugging and when checking the logs. This commit
also adds more robust signal handlers - to handle exceptions during
script execution. These handlers won't mask some crashes and it
also unifies exit codes between different scripts. These changes
have already been helpful to debugging "bypass" flag handling.
2022-05-18 01:22:29 +02:00
Julius Goryavsky
b081ad8c65 MDEV-28423: Galera IST is failing on Joiner node
This commit fixes an issue with IST handling in
version 10.9 which is a regression after MDEV-26971
and related to trying to get a non-existent "total"
tag on the IST branch (this tag is only defined in
SST mode).
2022-05-18 01:22:29 +02:00
Hartmut Holzgraefe
480323f7d6 MDEV-19161: Let galera_new_cluster use "restart" instead of "start"
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2022-05-12 07:46:26 +03:00
Julius Goryavsky
fccca49997 MDEV-28377: galera_as_slave_nonprim bind: Address already in use
This commit fixes a crash reported as MDEV-28377 and a number
of other crashes in automated tests with mtr that are related
to broken .cnf files in galera and galera_3nodes suites, which
happened when automatically migrating MDEV-26171 from 10.3 to
subsequent higher versions.
2022-04-26 20:44:20 +02:00
Marko Mäkelä
394784095e Merge 10.3 into 10.4 2022-04-21 11:33:59 +03:00
Julius Goryavsky
42908dc5fb MDEV-26171: wsrep_sst_receive_address does not parse IPv6 address correctly
This commit fixes problems with parsing ipv6 addresses given via
the wsrep_sst_receive_address and wsrep_node_address options.

Also, this commit removes extra lines in the configuration files
in the mtr test suites for Galera related to these parameters.
2022-04-12 17:14:39 +02:00
Julius Goryavsky
7a03128faf MDEV-28205: SST via mariabackup stops on failure while archiving logs
Currenly SST script for mariabackup stops on any failure while archiving
logs, e.g. when unable to create directory, insufficient permissions, gzip
failure, etc. However, in case of such problems, the script should issue
a warning and continue without archiving, but not exit with a fatal error.

This commit adds this fix to the SST script for mariabackup.
2022-04-07 13:26:36 +02:00
Marko Mäkelä
d172df9913 MDEV-25975: Merge 10.3 into 10.4 2022-04-06 09:18:38 +03:00
Marko Mäkelä
d6d66c6e90 Merge 10.3 into 10.4 2022-04-06 08:59:09 +03:00
Marko Mäkelä
e9735a8185 MDEV-25975 innodb_disallow_writes causes shutdown to hang
We will remove the parameter innodb_disallow_writes because it is badly
designed and implemented. The parameter was never allowed at startup.
It was only internally used by Galera snapshot transfer.
If a user executed
SET GLOBAL innodb_disallow_writes=ON;
the server could hang even on subsequent read operations.

During Galera snapshot transfer, we will block writes
to implement an rsync friendly snapshot, as follows:

sst_flush_tables() will acquire a global lock by executing
FLUSH TABLES WITH READ LOCK, which will block any writes
at the high level.

sst_disable_innodb_writes(), invoked via ha_disable_internal_writes(true),
will suspend or disable InnoDB background tasks or threads that could
initiate writes. As part of this, log_make_checkpoint() will be invoked
to ensure that anything in the InnoDB buf_pool.flush_list will be written
to the data files. This has the nice side effect that the Galera joiner
will avoid crash recovery.

The changes to sql/wsrep.cc and to the tests are based on a prototype
that was developed by Jan Lindström.

Reviewed by: Jan Lindström
2022-04-06 08:06:49 +03:00
Monty
c4ebb2bd04 Fixed that mysql_upgrade doesn't give errors about mariadb.sys
The reason for this fix was that when I tried to run mysql_upgrade
at home to update an old 10.5 installation, mysql_upgrade failed
with warnings about mariadb.sys user not existing.

If the server was started with --skip-grants, there would be no warnings
from mysql_upgrade, but in some cases running mysql_upgrade again could
produce new warnings.

The reason for the warnings was that any access of the mysql.user view
will produce a warning if the mariadb.sys user does not exists.

Fixed with the following changes:
- Disable warnings about mariadb.sys user not existing
- Don't overwrite old mariadb.sys entries in tables_priv and global_priv
- Ensure that tables_priv has an entry for mariadb.sys if the user exists.
  This fixes an issue that tables_priv would not be updated if there
  was a failure directly after global_priv was updated.
2022-04-04 16:14:22 +03:00