This commit fixes a bug where IST could be rejected in favor of SST
when ssl-mode=VERIFY_CA and when mariabackup is used. It also contains
a test and small code simplifications that will make it easier to find
bugs in the future.
The existing INFORMATION_SCHEMA.TABLE_PRIVILEGES displays only those
privileges that were specifically granted on the table level,
whereas it may be useful to see privileges granted at the database
and global level.
This commit adds a new view `table_privileges` to the `sys` schema
for that purpose. The view shows privileges on existing tables
and views, combining all possible levels:
- user_privileges
- schema_privileges
- table_privileges
Fix old_mode flags conflict between OLD_MODE_NO_NULL_COLLATION_IDS
and OLD_MODE_LOCK_ALTER_TABLE_COPY.
Both flags used to be 1 << 6, now OLD_MODE_LOCK_ALTER_TABLE_COPY changed
to be 1 << 7
Without pam compiled there will be no auth_pam_tool_dir, so check this
before attempting something that will error.
Reviewer: Sergei Golubchik / Daniel Black
Due to the different command line format of the timeout
utility on FreeBSD and Linux, SST scripts for mariabackup
may not work on FreeBSD. This commit fixes this problem
by adding a different command to test options on FreeBSD
and adding proper formatting for the utility options.
The MDEV-29693 conflict resolution is from Monty, as well as is
a bug fix where ANALYZE TABLE wrongly built histograms for
single-column PRIMARY KEY.
Also includes a fix for safe_malloc error reporting.
Other things:
- Copied main.log_slow from 10.4 to avoid mtr issue
Disabled test:
- spider/bugfix.mdev_27239 because we started to get
+Error 1429 Unable to connect to foreign data source: localhost
-Error 1158 Got an error reading communication packets
- main.delayed
- Bug#54332 Deadlock with two connections doing LOCK TABLE+INSERT DELAYED
This part is disabled for now as it fails randomly with different
warnings/errors (no corruption).
SST for mariabackup may not destroy old files if datadir or
other working directory is declared as a symlink due to the lack
of the "-L" option among the find utility options, similarly SST
for rsync in some cases may not transfer data directories if they
are created as symlinks. This fix adds the missing option and
generally unifies the work with find utility options to avoid
failures in the interpretation of directories and regular
expressions.
Before starting to go over the format string, prepare the current time
zone information incase '%z' or '%Z' is encountered.
This information can be obtained as given below:
A) If timezone is not set ( meaning we are working with system timezone):
Get the MYSQL_TIME representation for current time and GMT time using
current thread variable for timezone and timezone variable for UTC
respectively. This MYSQL_TIME variable will be used to calculate time
difference. Also convert current time in second to tm structure to
get system timezone information.
B) If timezone is set as offset:
Get timezone information using current timezone information and store
in appropriate variable.
C) If timezone is set as some place (example: Europe/Berlin)
Get timezone information by searching the timezone. During internal
timezone search, information like timeoffset from UTC and abbrevation
is stored in another relevant structure. Hence use the same information.
.snapshot exists as a directory on NetApp storage and
should not be copied during the sst process.
Thanks Daniel Czadek for the bug report.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
The table structure from MySQL-5.1.14 is:
CREATE TABLE `slow_log` (
`start_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`user_host` mediumtext NOT NULL,
`query_time` time NOT NULL,
`lock_time` time NOT NULL,
`rows_sent` int(11) NOT NULL,
`rows_examined` int(11) NOT NULL,
`db` varchar(512) DEFAULT NULL,
`last_insert_id` int(11) DEFAULT NULL,
`insert_id` int(11) DEFAULT NULL,
`server_id` int(11) DEFAULT NULL,
`sql_text` mediumtext NOT NULL
) ENGINE=CSV DEFAULT CHARSET=utf8 COMMENT='Slow log'
Even as far back as MySQL-5.5.40 this table could be created as NULLs
where not permitted in the CSV table time, but it seems they
where allowed sometime.
As the first part of mariadb-upgrade adds the column thread_id without
correcting the 'NULL'able status of existing columns it fails.
We reorder the sql statements in the ugprade as follows:
ALTER TABLE slow_log MODIFY {columns} {new types} NOT NULL,....
As thread_id doesn't exist in the above statement it was removed from
the first ALTER TABLE statement to prevent failure.
Previous ALTER TABLE slow_log where moved later appending thread_id
and rows_affected, and also enforces the type of thread_id if it
was incorrectly like the now first ALTER STATEMENT slow_log used
to do.
mariadb-install-db --auth-root-authentication-method=normal created 4
root accounts by default, but only two of these had PROXY privilege
granted.
mariadb-install-db (default option
--auth-root-authentication-method=socket) as non-root user also didn't
grant PROXY priv to the created nonroot@localhost user.
To fix this, in mysql_system_tables_data.sql, we re-use tmp_user_nopasswd
as this contains the list of all root users.
REPLACE INTO tmp_proxies_priv SELECT @current_hostname, IFNULL(@auth_root_socket, 'root')
creates the $user@$current_host but will not error if @auth_root_socket
is null. Note @current_hostname lines are filtered out with
--cross-bootstrap in mariadb-install-db so it was needed to include this
expression for consistency.
Like the existing mysql_system_tables.sql is used to create teh
$user@localhost proxies_priv.
Test cases roles.acl_statistics, perfschema,privilege_table_io depends on the number of proxy users.
After:
--auth-root-authentication-method=normal:
MariaDB [mysql]> select * from global_priv;
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
| Host | User | Priv |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
| localhost | mariadb.sys | {"access":0,"plugin":"mysql_native_password","authentication_string":"","account_locked":true,"password_last_changed":0} |
| localhost | root | {"access":18446744073709551615} |
| bark | root | {"access":18446744073709551615} |
| 127.0.0.1 | root | {"access":18446744073709551615} |
| ::1 | root | {"access":18446744073709551615} |
| localhost | | {} |
| bark | | {} |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------+
7 rows in set (0.001 sec)
MariaDB [mysql]> select * from proxies_priv;
+-----------+------+--------------+--------------+------------+---------+---------------------+
| Host | User | Proxied_host | Proxied_user | With_grant | Grantor | Timestamp |
+-----------+------+--------------+--------------+------------+---------+---------------------+
| localhost | root | | | 1 | | 2023-07-10 12:12:24 |
| 127.0.0.1 | root | | | 1 | | 2023-07-10 12:12:24 |
| ::1 | root | | | 1 | | 2023-07-10 12:12:24 |
| bark | root | | | 1 | | 2023-07-10 12:12:24 |
+-----------+------+--------------+--------------+------------+---------+---------------------+
--auth-root-authentication-method=socket:
MariaDB [mysql]> select * from proxies_priv;
+-----------+------+--------------+--------------+------------+---------+---------------------+
| Host | User | Proxied_host | Proxied_user | With_grant | Grantor | Timestamp |
+-----------+------+--------------+--------------+------------+---------+---------------------+
| localhost | root | | | 1 | | 2023-07-10 12:11:55 |
| localhost | dan | | | 1 | | 2023-07-10 12:11:55 |
| bark | dan | | | 1 | | 2023-07-10 12:11:55 |
+-----------+------+--------------+--------------+------------+---------+---------------------+
3 rows in set (0.017 sec)
MariaDB [mysql]> select * from global_priv;
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Host | User | Priv |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| localhost | mariadb.sys | {"access":0,"plugin":"mysql_native_password","authentication_string":"","account_locked":true,"password_last_changed":0} |
| localhost | root | {"access":18446744073709551615,"plugin":"mysql_native_password","authentication_string":"invalid","auth_or":[{},{"plugin":"unix_socket"}]} |
| localhost | dan | {"access":18446744073709551615,"plugin":"mysql_native_password","authentication_string":"invalid","auth_or":[{},{"plugin":"unix_socket"}]} |
| localhost | | {} |
| bark | | {} |
+-----------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.000 sec)
MariaDB [mysql]> show grants;
+----------------------------------------------------------------------------------------------------------------------------------------+
| Grants for dan@localhost |
+----------------------------------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO `dan`@`localhost` IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket WITH GRANT OPTION |
| GRANT PROXY ON ''@'%' TO 'dan'@'localhost' WITH GRANT OPTION |
+----------------------------------------------------------------------------------------------------------------------------------------+
* IS_USER_TEMP_TABLE() was misleading, name didn't match the code
* list of temp tables was rescanned number_of_databases times
* some temporary tables were not shown (from nonexistent databases)
* some temporary tables were shown more than once (e.g. after self-joins)
* sys.table_exists() - avoid querying I_S twice
* fix handling of temporary MERGE tables - it's pointless to fully open
them, they're not in thd->temporary_tables, so they simply fail to
open and are skipped. Relax the assertion instead.
- MDEV-28342 raised the error in case temporary table shadows base table
- Now we are allowed to shadow base tables with temporary tables and
`sys.create_synonym_db()` can easily check for existance of temporary table and
ignore view creation, since it is not supported to create view from
temporary table.
Reviewed-by: <monty@mariadb.org>,
<vicentiu@mariadb.org>
This commit updates sysschema to work with the new behaviour of show
tables and information_schema.tables table showing temporary tables for
current connection.
Co-authored-by: Monty <monty@mariadb.org>
Reviewer: <vicentiu@mariadb.org>
mysql.proc. The table is probably corrupted"
Analysis: When mysql_upgrade runs statements for upgrade, characterset is
converted to utf8mb4 because server starts with old_mode that interprets
utf8 to utf8mb4, but mysql.proc table has "utf8mb3" as hardcoded, so
it crashes with corrupted table.
Fix: Changed Table_check_intact::check() definition to allow both
utf8mb3 and utf8mb4 by checking prefix and changing the upgrade scripts
to explicitly use utf8mb3
- Moved view checks after privilege tables are fixed. This is to avoid
warnings about wrongly defined mysql.proc when checking views.
- Don't use stat tables before they have been fixed.
- Don't run mysql_fix_view() if 'FOR MYSQL' is used if the view is
already a MariaDB view.
- Added 'FOR UPGRADE' as an option for 'REPAIR VIEW' to be able to
detect if the REPAIR command comes from mariadb_upgrade. In this
case we get a warning, instead of an error, if a definer of a view
does not exists.
This commit adds a new 'no-sni' option to socat which is required to
properly authenticate with newer socat versions (after version 1.7.4+).
This option is needed to disable the automatic use of the SNI feature
(Server Name Indication) since the SST script directly specifies the
commonname if necessary and automatic activation of the SNI feature
is unnecessary in such scenarios.
These are mainly internal files so is a low impact change.
The few scripts/mysql*sql where renames to mariadb_* prefix
on the name.
mysql-test renamed to mariadb-test in the final packages
* it isn't "pfs" function, don't call it Item_func_pfs,
don't use item_pfsfunc.*
* tests don't depend on performance schema, put in the main suite
* inherit from Item_str_ascii_func
* use connection collation, not utf8mb3_general_ci
* set result length in fix_length_and_dec
* do not set maybe_null
* use my_snprintf() where possible
* don't set m_value.ptr on every invocation
* update sys schema to use the format_pico_time()
* len must be size_t (compilation error on Windows)
* the correct function name for double->double is fabs()
* drop volatile hack
Commit reduces need of AWK-command at least
for Debian mariadb-server-compat package.
Commit removes need of AWK-command from
scripts/mysql_install_db.sh script.
AWK command is replace by purely Posix sh compiliant version.
In some cases, the errors would not be written to the log.
This was however not critical as in most cases mysql_install_db
should not normally write anything to the log.
Renaming the default MariaDB backup directory from
xtrabackup_backupfiles to mariadb_backup_files.
Renaming files:
- xtrabackup_binlog_info to mariadb_backup_binlog_info
- xtrabackup_checkpoints to mariadb_backup_checkpoints
- xtrabackup_galera_info to mariadb_backup_galera_info
- xtrabackup_info to mariadb_backup_info
- xtrabackup_slave_info to mariadb_backup_slave_info
- Change to use 'mariadbd' instead of 'mysqld' in help texts and other
visible places.
- Start binary 'mariadbd' instead of 'mysqld'. This will remove a warning
in 11.0 when running mysql_install_db.
- Use my_print_defaults --mariadbd instead of --mysqld
- Use --skip-log-error if the user don't have access to log-error file.
This it needed to allow mysql_install_db to work silenty for users that
has not write access to /var/log.
Other things:
- Updated my_print_defaults to support --mariadbd
If you are running mariadb-install-db from a source tree instead of
installation it was executing `mysqld` instead of `mariadb` which showed
the deprecation warning. This patch fixes that as well as fixing
messages and links to other things that have been renamed.
- Remove DBUG calls from my_winfile.c where call and parameters
are already printed by mysys.
- Remove DBUG from my_get_osfhandle() and my_get_open_flags() to remove
DBUG noise.
- Updated convert-debug-for-diff to take into account windows.
- Changed some DBUG_RETURN(function()) to tmp=function(); DBUG_RETURN(tmp);
This is needed as Visual C++ prints for DBUG binaries a trace for
func_a()
{
DBUG_ENTER("func_a");
DBUG_RETURN(func_b())
}
as
>func_a
<func_a
>func_b
<func_b
instead of when using gcc:
>func_a
| >func_b
| <func_b
<func_a
Running mysql_upgrade should end up with the exact same system tables as fresh
installations have after running mysql_install_db. To ensure the upgrade is
correct and complete:
- Remove the redundant modification of thread_id`. On 5.5 version, the
`general_log` table was created as `CREATE TABLE IF NOT EXISTS general_log
(..., thread_id INTEGER NOT NULL, ...)`, and starting from 10.0+, the table is
created as `CREATE TABLE IF NOT EXISTS general_log (..., thread_id BIGINT(21)
UNSIGNED NOT NULL, ...)`, but mysql_upgrade is not properly upgrading the
table. It modifies the `thread_id` twice in one query, which could leave the
table not modified and lead to other potential error when upgrading from
MariaDB 5.5 or older.
- Update `servers` to ensure `Host` and `User` has correct data type if
upgrading from 10.1 or older. On versions 10.0 and 10.1, the `servers` table
was created as `CREATE TABLE IF NOT EXISTS servers (..., Host char(64) NOT
NULL DEFAULT , ..., Owner char(64) NOT NULL DEFAULT , ...)`, and starting
from 10.2, the table is created as `CREATE TABLE IF NOT EXISTS servers (...,
Host varchar(2048) NOT NULL DEFAULT , ..., Owner varchar(512) NOT NULL
DEFAULT , ...)`.
All new code of the whole pull request, including one or several files that
are either new files or modified ones, are contributed under the BSD-new license.
I am contributing on behalf of my employer Amazon Web Services, Inc.
For both Deb and RPM, create mariadb-client-compat and
mariadb-server-compat containing the mysql links to the mariadb
named executables/scripts.
The mariadb-client-core mysqlcheck was moved to mariadb-client-compat.
The symlinks in MYSQL_ADD_EXECUTABLE is tagged as a
{Client,Server}Symlinks component and placed in
the symlinks packages.
Man pages are restructured be installed into compat package
if that matches the executable.
Columnstore has a workaround as it doesn't use the cmake/plugin.cmake.
Scripts likewise have compatibility symlinks are in
the {server,client}-compat packages.
Co-author: Andrew Hutchings <andrew@linuxjedi.co.uk>
Closes#2390
don't include my_progname in the error message, my_error starts from it
automatically, resulting in, like
/usr/bin/mysqladmin: Notice: /usr/bin/mysqladmin is deprecated and will be removed in a future release, use command 'mariadb-admin'
and remove "Notice" so that the problem description would directly
follow the executable name.
make the check to work when the executable is in the PATH
(so, invoked simply like 'mysql' and thus readlink cannot find it)
fix the check in mysql_install_db and mysql_secure_installation to not
print the warning if the intermediate path contains "mysql" substring
add this message also to
* mysql_waitpid
* mysql_convert_table_format
* mysql_find_rows
* mysql_setpermissions
* mysqlaccess
* mysqld_multi
* mysqld_safe
* mysqldumpslow
* mysqlhotcopy
* mysql_ldb
Closes#2273
Eventually mysql symlinks will go away, as MariaDB and MySQL keep
diverging and we do not want to make it impossible to install
MariaDB and MySQL side-by-side when users want it.
It also useful if people start using MariaDB tools with MariaDB.
If the exe doesn't begine with "mariadb" or is a symlink,
print a warning to use the resolved name.
In my_readlink, add check on my_thread_var as its used by comp_err
and other build utils that also use my_init.
Renames the upgrade state file, and ensures the old
file is properly removed when `mariadb-upgrade` tool is executed.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
try to make them less confusing for users.
Hopefully, if the version string will be changed like
- mariadb Ver 15.1 Distrib 10.11.2-MariaDB for Linux (x86_64)
+ mariadb from 10.11.2-MariaDB, client 15.1 for Linux (x86_64)
users will be less inclined to reply "15.1" to the question
"what mariadb version are you using?"
This commit adds support for the --aria-log-dir-path
option on the command line and for the aria-log-dir-path
option in the configuration file to the SST scripts, since
before this change these parameters were completely ignored
during SST - SST scripts assumed that aria logs files are
always located in the same directory as logs for innodb.
Tests for this change will be added as a separate commit,
along with tests for MDEV-30157 and MDEV-28669.
This fix adds separate handling for "undo*" files that contain undo
logs as part of innodb files and adds a filter for undo* to the main
filter used when initially transferring files with rsync.
This commit adds even more correct handling of parameters
with paths when they contain leading or trailing spaces and/or
slashes. Also it fixes problems that occur when the user specified
explicit paths to additional directories, but these paths match
the specified path of the data directory - in this case, additional
subdirectories should be treated (in relation to the data directory)
in the same way as if these paths were not specified or as if they
are implicitly specified as "." or "./". But prior to this fix,
existing code treated any values as if they were completely
separate directories, whether or not they actually point to the
same location to which datadir points to - and this sometimes
resulted in incorrect file transfers.
This fix does not contain separate tests, as tests will be
part of the main commit(s). This fix has been made as a separate
commit to facilitate review for major substantive fixes related
to MDEV-30157 and MDEV-28669.
The MDEV-25004 test innodb_fts.versioning is omitted because ever since
commit 685d958e38 InnoDB would not allow
writes to a database where the redo log file ib_logfile0 is missing.
If mariadb-service-convert is run and the user variable is unset then
this sets `User=` in `[Service]`, which then tries to run mariadb as
root, which in-turn fails. This only happens when mysqld_safe is missing
which is all the time now. So don't set `User=` if there is no user variable.
Reviewer: Sergei Golubchik <serg@mariadb.org> (in PR #2382)
* mariadb-service-convert to use mariadbd-safe
* galera_recovery to use mariadbd
* mtr - wsrep use mariadb executables
* debian/mariadb-server.mariadb.init use mariadbd-safe
* debian/tests/smoke uses mariadb instead of mysql as client.
Co-Author: Daniel Black <daniel@mariadb.org>
Previously we parsed it out in mysql_install_db for use in the error
message, but failed to pass it to mysqld in the bootstrap.
Also match log_error as it might appear in the .cnf files.
Thanks Michal Schorm for the test case.
Reviewed by: Faustin Lammler
MDEV-16735 describes how mysql_upgrade fails when alter_algorithm
is set to a value different than 'DEFAULT'/'COPY'. It was marked as
fixed by 0ee0868, but the fix didn't covered the possibility of having
the global value of alter_algorithm set to something different than
'DEFAULT'/'COPY'. To ensure that the upgrade process works properly
regardless the global value of alter_altorithm, this commit force it's
value to 'DEFAULT' (note the quotes) for the mysql_upgrade session.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
The mysqld_multi script template used @datadir@ as default log
destination, this is not the MariaDB datadir in this context though
but rather the -- typically write-only -- /share dir.
The correct placeholder to use here is @localstatedir@ which gets
replaced with the actual MariaDB datadir
Co-authored-by: Hartmut Holzgraefe <hartmut@php.net>
Increasing the data type from char(32) to char(64) for
the following system columns:
- mysql.event.collation_connection
- mysql.event.db_collation
- mysql.proc.collation_connection
- mysql.proc.db_collation
This change was forgotten during MDEV-27009.
Also fixing the comment "Start/End of 10.9 tests" in ctype_utf8_uca.test
to "Start/End of 10.10 tests", as MDEV-27009 was actually
released in 10.10.
The mysqld_safe script was using bad grep options.
The line that was fixed was likely meant to be 2 separate grep commands,
piped into each other, with each one removing any lines that matched.
The `-E` option was unneeded, as the command is not using regex.
This commit restores defaults and functionality regarding binlogs
to the way it was prior to MDEV-27524. The mariabackup utility no
longer saves binlogs files as part of a backup without the --galera-info
option. However, since we use --galera-info during SST, the behavior
of mariabackup changes and, in combination with GTIDs support enabled,
mariabackup transfers one (most recent) binlog file obtained after
FLUSH BINARY LOGS. In other cases, binlogs are not transferred during
SST in mariabackup mode. As for SST in the rsync mode, it works the
same way as before MDEV-27524 - by default it transfers one last
binlog file.
The --sst-max-binlogs option for mariabackup and the sst_max_binlogs
parameter in the [sst] / server sections are no longer supported for
SST via mariabackup.
The incorrect type of mysql.column_stats caused the server during the
upgrade of every other table to complain:
[ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9
and expected column 'histogram' at position 10 to have type longblob.
To prevent these verbose server errors, we upgrade the
mysql.column_stats table first.
Consequently limit "Incorrect definition of table mysql.*" to the appropriate
set of limited test cases.
The rpl_gtid_errorhandling.result changes the GTID number by one
because of the added early suppression (adding a table row).
Reviewer: Vicențiu Ciorbaru
FixesMariaDB/mariadb-docker#438
This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).
Also fixed broken logic of automatic generation of the Diffie-Hellman parameters
for socat version less than 1.7.3 (which defaults to 512-bit values instead of
2048-bit ones).
