MDEV-23201 correctly reordered columns in mysql.user table when upgrading
from MySQL-5.7
Here we also correctly reorder columns in mysql.user table from an invalid order
caused by an upgrade from MySQL-5.7 to MariaDB-before-MDEV-23201.
In galera_3nodes.galera_safe_to_bootstrap node restart can happen too soon, when earlier SST joiner process is still active in the node.
Similar issue may hurt other mtr tests as well.
This is second variant of fix for this issue. Here we only change rsync SST script to wait a little bit if lingering SST rsync is observed to be in execution.
We assume that the previous mysqld and SST processes have been already signaled to abort during earlier stataup attempt.
If other SST methods (than rsync) suffer from similar overlapping SST execution, they should be sorted out separately within each SST method handler scripts.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
To fix this, it is necessary to add an option to exclude the
database with the name "lost+found" from processing (the database
name will be checked by the check_if_skip_database_by_path() or
by the check_if_skip_database() function, and as a result
"lost+found" will be skipped).
In addition, it is necessary to slightly modify the verification
logic in the check_if_skip_database() function.
Also added a new test galera_sst_mariabackup_lost_found.test
- This commit is based on patch 84fe9720a4d2483ff67b6a and suggestion of
Jean Weisbuch to use `ldconfig -p`
- Format of ldconfig output:
"libjemalloc.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libjemalloc.so.1"
```
$ ./scripts/mysqld_safe
201013 13:36:50 mysqld_safe Adding '/usr/lib/x86_64-linux-gnu/libjemalloc.so.1' to LD_PRELOAD for mysqld
201013 13:36:50 mysqld_safe Logging to '/home/anel/builds/data-10.5/mysqld_safe_anel.err'.
201013 13:36:50 mysqld_safe Starting mysqld daemon with databases from /home/anel/builds/data-10.5
```
Reviewed by: Daniel Black, Faustin Lammler
This patch fixes several flaws in the SST scripts that cause
failures while running tests that use version 6 IP addresses
for cluster nodes.
First, if the netcat utility is used for streaming (but not socat),
then in accordance with its command line syntax, we need to remove
the square brackets around the IPv6 address. However, for socat,
the address must contain square brackets, as before.
Secondly, if an IPv6 address is used, then from the joiner side for
a number of systems (such as Debian) we need to explicitly specify
the "-6" option, otherwise a listening socket with an IPv6 address
may not be created.
This patch also contains code improvements in the wsrep_sst_common.
Changed the code that pars the connection address - fixed the
shortcomings that sometimes led to incorrect parsing of parameters
when using shells other than the latest versions of bash.
Also, this patch removes the duplicate code that is intended
for parsing the connection address and which was located in the
wsrep_sst_mariabackup file, since all the necessary actions have
already been done in wsrep_sst_common and there they are done in
such a way that any shell is supported, not just bash.
The fix does not require separate tests, since all the
necessary tests are already present in the galera_3nodes suite.
On the contrary, after this fix, tests using IPv6 addresses can
be removed from the disabled list (this will be done in a separate
commit related to MDEV-23659).
As MariaDB tables are expected in a fixed order, the cross upgrade
previously placed roles, default_role and max_statement_time after
now unused mysql-5.7 columns.
Test:
$ cp -a /usr/local/mysql-5.7.31/data/ /tmp/m57data
$ sql/mysqld --no-defaults --skip-networking --datadir=/tmp/m57data --socket=/tmp/${PWD##*/}.sock --verbose --lc-messages-dir=$PWD/sql/share
2020-10-02 11:02:05 140135193212864 [Note] sql/mysqld (mysqld 10.2.34-MariaDB) starting as process 1457667 ...
2020-10-02 11:02:05 140135193212864 [Note] InnoDB: Mutexes and rw_locks use GCC atomic built
$ client/mysql_upgrade --no-defaults -u root -pbob -S /tmp/build-mariadb-server-10.2.sock
MySQL upgrade detected
Phase 1/7: Checking and upgrading mysql database
Processing databases
mysql
mysql.columns_priv OK
mysql.db OK
mysql.engine_cost OK
mysql.event OK
mysql.func OK
mysql.gtid_executed OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.ndb_binlog_index OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.server_cost OK
mysql.servers OK
mysql.slave_master_info OK
mysql.slave_relay_log_info OK
mysql.slave_worker_info OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.user OK
Upgrading from a version before MariaDB-10.1
Phase 2/7: Installing used storage engines
Result:
| user | CREATE TABLE `user` (
`Host` char(60) COLLATE utf8_bin NOT NULL DEFAULT '',
`User` char(80) COLLATE utf8_bin NOT NULL DEFAULT '',
`Password` char(41) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL DEFAULT '',
`Select_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Insert_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Update_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Delete_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Create_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Drop_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Reload_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Shutdown_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Process_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`File_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Grant_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`References_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Index_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Alter_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Show_db_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Super_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Create_tmp_table_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Lock_tables_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Execute_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Repl_slave_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Repl_client_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Create_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Show_view_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Create_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Alter_routine_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Create_user_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Event_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Trigger_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`Create_tablespace_priv` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`ssl_type` enum('','ANY','X509','SPECIFIED') CHARACTER SET utf8 NOT NULL DEFAULT '',
`ssl_cipher` blob NOT NULL,
`x509_issuer` blob NOT NULL,
`x509_subject` blob NOT NULL,
`max_questions` int(11) unsigned NOT NULL DEFAULT 0,
`max_updates` int(11) unsigned NOT NULL DEFAULT 0,
`max_connections` int(11) unsigned NOT NULL DEFAULT 0,
`max_user_connections` int(11) NOT NULL DEFAULT 0,
`plugin` char(64) CHARACTER SET latin1 NOT NULL DEFAULT '',
`authentication_string` text COLLATE utf8_bin NOT NULL,
`password_expired` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`is_role` enum('N','Y') CHARACTER SET utf8 NOT NULL DEFAULT 'N',
`default_role` char(80) COLLATE utf8_bin NOT NULL DEFAULT '',
`max_statement_time` decimal(12,6) NOT NULL DEFAULT 0.000000,
`password_last_changed` timestamp NULL DEFAULT NULL,
`password_lifetime` smallint(5) unsigned DEFAULT NULL,
`account_locked` enum('N','Y') COLLATE utf8_bin NOT NULL DEFAULT 'N',
PRIMARY KEY (`Host`,`User`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Users and global privileges' |
MariaDB [(none)]> CREATE ROLE `aRole`;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> SET ROLE `aRole`;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> SET ROLE `aRole`;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> SELECT `User`, `is_role` FROM `mysql`.`user`;
+---------------+---------+
| User | is_role |
+---------------+---------+
| root | N |
| mysql.session | N |
| mysql.sys | N |
| dan | N |
| aRole | Y |
+---------------+---------+
5 rows in set (0.00 sec)
Reviewer: Anel Husakovic
Leave debian/additions/mysqlreport as #!/usr/bin/perl
Acknowledge that `env perl` is a hack, a complete fix
needs to consider which path perl is at and insert into
these scripts.
The usefulness of these scripts is questionable.
Adding any unknown option to the "[mysqld_safe]" section makes
mysqld impossible to start with mysqld_multi. For example, after
adding the unknown option "numa_interleave" to the "[mysqld_safe]"
section, mysqld_multi exits with the following diagnostics:
[ERROR] /usr/local/mysql/bin/mysqld: unknown option '--numa_interleave'
To get rid of this behavior, this patch by default adds the "--loose-"
prefix to all unknown (for mysqld_safe) options. This behavior can be
enabled explicitly with the --ignore-unknown option and disabled with
the --no-ignore-unknown option.
The problem is caused by the fact that adding the
--defaults-group-suffix option to fix MDEV-18863 causes
mysqld to read all options from the appropriate sections
of the config file, including options specific to mysqld_multi.
Reading unknown options (which are not supported by mysqld)
causes mysqld to terminate with an error.
However, the MDEV-18863 problem has been completely fixed
by passing options on the command line, and now there is no
need to specify the --defaults-group-suffix option (we just
need to give priority to options passed through the command
line, so as not to break MDEV-18863).
On FreeBSD, perl isn't in /usr/bin, its in /usr/local/bin or
elsewhere in the path.
Like storage/{maria/unittest/,}ma_test_* , we use /usr/bin/env to
find perl and run it.
The problem is caused by the operation of netcat streamer and does
not appear on systems where socat is installed. We need to add the
"-N" option for netcat to call shutdown() on the socket when receiving
EOF from STDIN.
Also add a check for tmp file being empty and bail out with a clear
error message in such a case, as mysqld_safe prevents normal stderr
from being displayed anywhere and would fail silently on this.
Setting "streamfmt=mbstream" in the "[sst]" section causes SST to fail
because the format automatically switches to 'tar' by default (insead
of mbstream).
To fix this, we need to add mbstream to the list of valid values for
the format, making it synonymous with xbstream. This must be done both
in the SST script and when parsing the options of the corresponding
utilities.
When the mysqld_multi script passes the --defaults-group-suffix
option to mysqld, it must remove the initial substring with the
group name ("mysqld") from option value, because otherwise substring
"mysqld" will be added to the group name and then the group name
will contain the word "mysqld" twice, which is wrong, because
mysqld itself adds the suffix received to the group name.
When the mysqld_multi script passes the --defaults-group-suffix
option to mysqld, it must remove the initial substring with the
group name ("mysqld") from option value, because otherwise substring
"mysqld" will be added to the group name and then the group name
will contain the word "mysqld" twice, which is wrong, because
mysqld itself adds the suffix received to the group name.
A syntax error in the mysqld_multi.sh script has been fixed
here + a "--defaults-group-suffix" option has been moved to
the top of the mysqld options list.
A syntax error in the mysqld_multi.sh script has been fixed
here + a "--defaults-group-suffix" option has been moved to
the top of the mysqld options list.
mysqld_safe --dry-run needs to either call exit or return, depending if
it is being sourced or not, otherise return can lead to the error:
return: can only `return' from a function or sourced script
The original fix suggestion was proposed by FaramosCZ <mschorm@centrum.cz>
