Commit graph

8995 commits

Author SHA1 Message Date
Sergei Golubchik
3a33ae8601 MDEV-33091 pcre2 headers aren't found on Solaris
use pkg-config to find pcre2, if possible

rename PCRE_INCLUDES to use PKG_CHECK_MODULES naming, PCRE_INCLUDE_DIRS
2024-01-19 20:23:45 +01:00
Marko Mäkelä
2b01e5103d Merge 10.5 into 10.6 2023-12-19 18:41:42 +02:00
Marko Mäkelä
12995559f9 Merge 10.4 into 10.5 2023-12-19 18:30:58 +02:00
hsser
be694384d4 MDEV-31925 Fix for File Leak in mysql_upgrade with --check-if-upgrade-is-needed Option
This commit addresses the file leakage problem encountered with the mysql_upgrade --check-if-upgrade-is-needed command.
2023-12-19 19:30:02 +11:00
Marko Mäkelä
4ae105a37d Merge 10.4 into 10.5 2023-12-18 08:59:07 +02:00
Sergei Golubchik
e95bba9c58 Merge branch '10.5' into 10.6 2023-12-17 11:20:43 +01:00
Sergei Golubchik
a2c6d61db8 don't use dynstr_append() in mysqltest.cc
followup for ad796aaa94
2023-12-15 10:26:48 +01:00
Sergei Golubchik
98a39b0c91 Merge branch '10.4' into 10.5 2023-12-02 01:02:50 +01:00
Monty
18166e4501 mysqladmin -vv processlist now shows more information
What is shown in this case is the result from the following query:
select * from information_schema.processlist where id != connection_id()

This allows easy monitoring of for example MAX_MEMORY_USED
2023-11-30 18:40:43 +02:00
Oleksandr Byelkin
ad796aaa94 Remove unneeded strlen() calls. 2023-11-23 13:20:50 +01:00
Lena Startseva
9e701518e7 MDEV-32177: Add to the ps-protocol a comparison of the result
sets of the first and second execution. The results of the first
and second execution are compared only if result logging is enabled
Comparing two result sets is done as comparing two strings.

Added a new method 'read_stmt_results' to get a result set
after execution of prepare statement.
2023-11-23 13:20:31 +01:00
Oleksandr Byelkin
0ad4839429 Rename variables to make operation comprehansive 2023-11-23 13:19:47 +01:00
Marko Mäkelä
52ca2e65af Merge 10.5 into 10.6 2023-11-15 14:10:21 +02:00
Oleksandr Byelkin
9f83a8822f Merge branch '10.5' into mariadb-10.5.23 2023-11-14 08:41:23 +01:00
Oleksandr Byelkin
b83c379420 Merge branch '10.5' into 10.6 2023-11-08 15:57:05 +01:00
Oleksandr Byelkin
6cfd2ba397 Merge branch '10.4' into 10.5 2023-11-08 12:59:00 +01:00
Anel Husakovic
4eb8aeee8e MDEV-32462: mysql_upgrade -s still checks for non system tables
- Prevent opening of any user tables in case `upgrade-system-table`
option is used.
- Still there may be uninstalled data types in `mysql` system table so
allow it to perform.
- Closes PR 
- Reviewer: <daniel@mariadb.org>, <vicentiu@mariadb.org>
2023-10-30 21:33:06 +01:00
Oleg Smirnov
cbe61bf87f MDEV-30664 mysqldump output is truncated on Windows
Flush stdout on finalizing of mysqldump/mysqlbinlog output
to avoid truncation.

The same patch has been applied to the mysqltest.cc code with
  commit 34ff714b0d
  Author: Magnus Svensson <msvensson@mysql.com>
  Date:   Fri Nov 14 11:06:56 2008 +0100
    WL#4189 Make mysqltest flush log file at close if logfile is stdout

but not to mysqldump.c/mysqlbinlog.cc
2023-10-10 14:04:48 +07:00
Daniel Black
a34b989f0c MDEV-22418 mysqladmin wrong error with simple_password_check
mysqladmin's presumption about the cause of an error by looking
at the error code was presumptious. Server knows best, so pass
the error along.

Avoid returning -1 as a exit code, Linux makes this 255 and Windows
keeps this as -1.
2023-10-06 11:08:47 +03:00
Monty
4e9322e2ff MDEV-32203 Raise notes when an index cannot be used on data type mismatch
Raise notes if indexes cannot be used:
- in case of data type or collation mismatch (diferent error messages).
- in case if a table field was replaced to something else
  (e.g. Item_func_conv_charset) during a condition rewrite.

Added option to write warnings and notes to the slow query log for
slow queries.

New variables added/changed:

- note_verbosity, with is a set of the following options:
  basic            - All old notes
  unusable_keys    - Print warnings about keys that cannot be used
                     for select, delete or update.
  explain          - Print unusable_keys warnings for EXPLAIN querys.

The default is 'basic,explain'. This means that for old installations
the only notable new behavior is that one will get notes about
unusable keys when one does an EXPLAIN for a query. One can turn all
of all notes by either setting note_verbosity to "" or setting sql_notes=0.

- log_slow_verbosity has a new option 'warnings'. If this is set
  then warnings and notes generated are printed in the slow query log
  (up to log_slow_max_warnings times per statement).

- log_slow_max_warnings   - Max number of warnings written to
                            slow query log.

Other things:
- One can now use =ALL for any 'set' variable to set all options at once.
  For example using "note_verbosity=ALL" in a config file or
  "SET @@note_verbosity=ALL' in SQL.
- mysqldump will in the future use @@note_verbosity=""' instead of
  @sql_notes=0 to disable notes.
- Added "enum class Data_type_compatibility" and changing the return type
  of all Field::can_optimize*() methods from "bool" to this new data type.

Reviewer & Co-author: Alexander Barkov <bar@mariadb.com>
- The code that prints out the notes comes mainly from Alexander
2023-10-03 08:25:31 +03:00
Marko Mäkelä
0dd25f28f7 Merge 10.5 into 10.6 2023-09-11 14:46:39 +03:00
Marko Mäkelä
f8f7d9de2c Merge 10.4 into 10.5 2023-09-11 11:29:31 +03:00
Sergei Golubchik
4d96dba767 MDEV-25369 mysqlbinlog (mariadb-binlog) -T/--table option
clarify the help text
2023-09-06 22:38:41 +02:00
Oleksandr Byelkin
6bf8483cac Merge branch '10.5' into 10.6 2023-08-01 15:08:52 +02:00
Oleksandr Byelkin
65405308a1 Merge branch '10.4' into 10.5 2023-08-01 11:52:13 +02:00
Sergei Golubchik
ab10a675ac MDEV-31092 mysqldump --force doesn't ignore error as it should
failed SHOW CREATE FUNCTION means we don't dump this function,
but should still try to dump all other functions
2023-07-31 22:46:47 +02:00
Oleksandr Byelkin
f291c3df2c Merge branch '10.4' into 10.5 2023-07-27 15:43:21 +02:00
Lena Startseva
9854fb6fa7 MDEV-31003: Second execution for ps-protocol
This patch adds for "--ps-protocol" second execution
of queries "SELECT".
Also in this patch it is added ability to disable/enable
(--disable_ps2_protocol/--enable_ps2_protocol) second
execution for "--ps-prototocol" in testcases.
2023-07-26 17:15:00 +07:00
Oleksandr Byelkin
f52954ef42 Merge commit '10.4' into 10.5 2023-07-20 11:54:52 +02:00
Oleg Smirnov
6d911219d6 MDEV-30639 Upgrade to 10.8 and later does not work on Windows
During the upgrade procedure on Windows mysqld.exe is started with
the named pipe connection protocol. mysqladmin.exe then pings the
server to check if is up and running. Command line looks like:
   mysqladmin.exe --protocol=pipe --socket=mysql_upgrade_service_xxx ping
But the "socket" parameter resets the "protocol" which was previously
initialized with the "pipe" value, setting it to "socket".
As a result, connection cannot be established and the upgrade
procedure fails.
"socket" in Windows is used to pass the name of the pipe so resetting
the protocol is not valid in this case.

This commit fixes resetting of the "protocol" parameter with "socket"
parameter in the case when protocol has been previously initialized
to "pipe" value
2023-06-30 15:59:14 +07:00
Monty
d671fec431 Fixed some errors & warnings when running mariadb-upgrade on MySQL instance
- Moved view checks after privilege tables are fixed. This is to avoid
  warnings about wrongly defined mysql.proc when checking views.
- Don't use stat tables before they have been fixed.
- Don't run mysql_fix_view() if 'FOR MYSQL' is used if the view is
  already a MariaDB view.
- Added 'FOR UPGRADE' as an option for 'REPAIR VIEW' to be able to
  detect if the REPAIR command comes from mariadb_upgrade. In this
  case we get a warning, instead of an error, if a definer of a view
  does not exists.
2023-06-25 16:15:08 +03:00
Sergei Golubchik
d32fc5b8e0 MDEV-31461 mariadb SIGSEGV when built with -DCLIENT_PLUGIN_DIALOG=STATIC 2023-06-19 12:12:21 +02:00
Sergei Golubchik
69684f689c use correct collation_connection in --view
mysqltest should use the same collation_connection in the service
connection (that creates views) as in the main connection

this makes weight_string("aaa") to return the expected value in --view
and fixes main.func_str failure in --view
2023-06-02 17:51:40 +02:00
Monty
f272463b02 Cleanup of MDEV-14974: --port ignored for --host=localhost
The old code added to 10.6 was inconsisting in how TCP/IP and
socket connection was chosen. One got also a confusing warning
in some cases.

Examples:
> ../client/mysql --print-defaults
../client/mysql would have been started with the following arguments:
--socket=/tmp/mariadbd.sock --port=3307 --no-auto-rehash
> ../client/mysql
ERROR 2002 (HY000): Can't connect to local server through socket '/tmp/mariadbd.sock' (2)
> ../client/mysql --print-defaults
../client/mysql would have been started with the following arguments:
--socket=/tmp/mariadbd.sock --port=3307 --no-auto-rehash
> ../client/mysql --port=3333
WARNING: Forcing protocol to  TCP  due to option specification. Please explicitly state intended protocol.
ERROR 2002 (HY000): Can't connect to server on 'localhost' (111)
> ../client/mysql --port=3333 --socket=sss
ERROR 2002 (HY000): Can't connect to local server through socket 'sss' (2)
> ../client/mysql --socket=sss --port=3333
ERROR 2002 (HY000): Can't connect to local server through socket 'sss' (2)

Some notable things:
- One gets a warning if one uses just --port if config file sets socket
- Using port and socket gives no warning
- Using socket and then port still uses socket

This patch changes things the following ways:
If --port= is given on the command line, the the protocol is automatically
  changed to "TCP/IP".
- If --socket= is given on the command line, the protocol is automatically
  changed to "socket".
- The last option wins
- No warning is given if protocol changes automatically.
2023-04-27 11:34:26 +03:00
Marko Mäkelä
818d5e4814 Merge 10.5 into 10.6 2023-04-25 13:10:33 +03:00
Oleksandr Byelkin
1d74927c58 Merge branch '10.4' into 10.5 2023-04-24 12:43:47 +02:00
Marko Mäkelä
854e8b189e MDEV-28976 fixup: A better fix
do_shutdown_server(): After sending SIGKILL, invoke wait_until_dead().
Thanks to Sergei Golubchik for pointing out that the previous fix
does not actually work.
2023-04-19 15:53:26 +03:00
Marko Mäkelä
d665186477 MDEV-28976: mtr must wait for server to actually die
do_shutdown_server(): Call wait_until_dead() also when we are forcibly
killing the process (timeout=0). We have evidence that killing
the process may take some time and cause mystery failures in
crash recovery tests. For InnoDB, several failures were observed between
commit da094188f6 and
commit 0ee1082bd2
when no advisory file locking was being used by default.
2023-04-19 14:08:53 +03:00
Marko Mäkelä
5bada1246d Merge 10.5 into 10.6 2023-04-11 16:15:19 +03:00
Alexander Barkov
62e137d4d7 Merge remote-tracking branch 'origin/10.4' into 10.5 2023-04-05 16:16:19 +04:00
Lorna Luo
da73db2382 Make 'move_file' command more reliable in mysqltest
The tests innodb.import_tablespace_race, innodn.restart, and innodb.innodb-wl5522 move
the tablespace file between the data directory and the tmp directory specified by
global environment variables. However this is risky because it's not unusual that the
set tmp directory (often under /tmp) is mounted on another disk partition or device,
and 'move_file' command may fail with "Errcode: 18 'Invalid cross-device link.'"

To stabilize mysqltest in the described scenario, and prevent such
behavior in the future, let make_file() check both from file path and to
file path and make sure they are either both under MYSQLTEST_VARDIR or
MYSQL_TMP_DIR.

All new code of the whole pull request, including one or several files that
are either new files or modified ones, are contributed under the BSD-new license.
I am contributing on behalf of my employer Amazon Web Services, Inc.
2023-04-03 14:36:11 +02:00
Oleksandr Byelkin
ac5a534a4c Merge remote-tracking branch '10.4' into 10.5 2023-03-31 21:32:41 +02:00
Debjyoti
d575b07c86 MDEV-24453 Added support for a 5th --verbose parameter in mariadb-upgrade to show mysql results for mysql_fix_privilege_tables 2023-03-27 16:29:14 +02:00
Anel Husakovic
2f6bb9cda5 MDEV-30698 Cover missing test cases for mariadb-binlog options --raw [and] --flashback
- Adding test case for --raw without -R
- Adding unsuported combination of --raw and --flashback parameters and
covered with test case
2023-03-08 12:26:00 +00:00
Anel Husakovic
7300ab32cc Update handling of mysqlbinlog's die() function
- Update per Monty's review for commit 5328dbdaa237d97e443120f2e08437fb274198c2
2023-03-08 12:26:00 +00:00
Anel Husakovic
f0ab1a28c9 MDEV-30697: Memory leak detected when mariadb-binlog is used with option flashback 2023-03-08 12:26:00 +00:00
Monty
fb8c1762ad Ensure that mysqlbinlog frees all memory at exit 2023-03-08 12:26:00 +00:00
Hugo Wen
7bdd878ae4 Fix few vulnerabilities found by Cppcheck
While performing SAST scanning using Cppcheck against source code of
commit 81196469, several code vulnerabilities were found.

Fix following issues:

1. Parameters of `snprintf` function are incorrect.

   Cppcheck error:

       client/mysql_plugin.c:1228: error: snprintf format string requires 6 parameters but only 5 are given.

   It is due to commit 630d7229 introduced option `--lc-messages-dir`
   in the bootstrap command. However the parameter was not even given
   in the `snprintf` after changing the format string.

   Fix:
   Restructure the code logic and correct the function parameters for
   `snprintf`.

2. Null pointer is used in a `snprintf` which could cause a crash.

   Cppcheck error:

       extra/mariabackup/xbcloud.cc:2534: error: Null pointer dereference

   The code intended to print the swift_project name, if the
   opt_swift_project_id is NULL but opt_swift_project is not NULL.
   However the parameter of `snprintf` was mistakenly using
   `opt_swift_project_id`.

   Fix:
   Change to use the correct string from `opt_swift_project`.

3. Potential double release of a memory

   Cppcheck error:

       plugin/auth_pam/testing/pam_mariadb_mtr.c:69: error: Memory pointed to by 'resp' is freed twice.

   A pointer `resp` is reused and allocated new memory after it has been
   freed. However, `resp` was not set to NULL after freed.
   Potential double release of the same pointer if the call back
   function doesn't allocate new memory for `resp` pointer.

   Fix:
   Set the `resp` pointer to NULL after the first free() to make sure
   the same address is not freed twice.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
2023-03-02 14:38:24 +11:00
Anel Husakovic
bd0d7ea540 MDBF-534: Coverity scan: fix client folder
---------------------------------
File: `mysqltest`
---------------------------------
- Coverity (SIZEOF_MISMATCH):
  - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074863&mergedDefectId=972322
    Function `qsort` have to use size of element that is `uchar *`

- Coverity (REVERSE_INULL):
  - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074524&mergedDefectId=1519693&fileStart=3376&fileEnd=3625
    First check if null and then use `strlen`, not reversed.

- FALSE POSITIVES
  - Coverity (TAINTED_SCALAR):
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074760&mergedDefectId=1519321

  - Coverity (CHECKED_RETURN):
    - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074692&mergedDefectId=971714
    - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53072839&mergedDefectId=971715

  - Coverity (FORWARD_NULL):
    There is already issued DBUG_ASSERT(query_end) few lines before
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074002&mergedDefectId=971916&eventId=53074002-5

  - Coverity (OVERRUN):
    - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074470&mergedDefectId=1519697
    - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074862&mergedDefectId=1520391
      `uint64_max` and `SIZE_MAX` (max for `size_t`) are same as `count` argument
      for `memcmp`.

  - Coverity (RESOURCE_LEAK):
    - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074163&mergedDefectId=1519889&eventId=53074163-446

- INTENTION:
  - Coverity (SIZEOF_MISMATCH):
    - https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728385&defectInstanceId=53074650&mergedDefectId=1520109
      `len` argument is used only in printing so it is not making impact (may be removed as an alternative).
      In this example size of pointer (8B) is used, that is not the size of value that pointer points to.
2023-02-17 13:43:43 +00:00
Anel Husakovic
24911a34b3 MDBF-534: Coverity scan: fix client folder
---------------------------------
File: `mysqlslap`
---------------------------------
- Coverity (CHECKED_RETURN):
  https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728428&defectInstanceId=53073524&mergedDefectId=1520114

- FALSE POSITIVES
  - Coverity (DC.WEAK_CRYPTO) (`random()`):
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728428&defectInstanceId=53073112&mergedDefectId=1225806
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728428&defectInstanceId=53074491&mergedDefectId=1409160
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728428&defectInstanceId=53074151&mergedDefectId=1409180
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728428&defectInstanceId=53073799&mergedDefectId=1409183

  - Coverity (TAINTED_SCALAR):
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728428&defectInstanceId=53074667&mergedDefectId=1519586
  - Coverity (UNINT):
  - Coverity (BAD_FREE):
    https://scan5.scan.coverity.com/reports.htm#v58936/p10357/fileInstanceId=231728428&defectInstanceId=53074415&mergedDefectId=1520371
2023-02-17 13:43:43 +00:00