Commit graph

9389 commits

Author SHA1 Message Date
Vladislav Vaintroub
061adae9a2 MDEV-16944 Fix file sharing issues on Windows in mysqltest
On Windows systems, occurrences of ERROR_SHARING_VIOLATION due to
conflicting share modes between processes accessing the same file can
result in CreateFile failures.

mysys' my_open() already incorporates a workaround by implementing
wait/retry logic on Windows.

But this does not help if files are opened using shell redirection like
mysqltest traditionally did it, i.e via

--echo exec "some text" > output_file

In such cases, it is cmd.exe, that opens the output_file, and it
won't do any sharing-violation retries.

This commit addresses the issue by introducing a new built-in command,
'write_line', in mysqltest. This new command serves as a brief alternative
to 'write_file', with a single line output, that also resolves variables
like "exec" would.

Internally, this command will use my_open(), and therefore retry-on-error
logic.

Hopefully this will eliminate the very sporadic "can't open file because
it is used by another process" error on CI.
2024-04-17 16:52:37 +02:00
Marko Mäkelä
829cb1a49c Merge 10.5 into 10.6 2024-04-17 14:14:58 +03:00
Kristian Nielsen
16aa4b5f59 Merge from 10.4 to 10.5
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-04-15 17:46:49 +02:00
Sergei Golubchik
41296a07c8 Merge branch '10.5' into 10.6 2024-04-11 13:58:22 +02:00
Yuchen Pei
e0b6db2de7
MDEV-31609 Send initial values of system variables in first OK packet
Values of all session tracking system variables will be sent in the
first ok packet upon connection after successful authentication.

Also updated mtr to print session track info on connection (h/t Sergei
Golubchik) so that we can write mtr tests for this change.
2024-04-10 11:13:46 +10:00
Yuchen Pei
e32736ec59
MDEV-31609 mtr: only print session tracking from the last statement
We do this by checking server status. By doing this we avoid printing
session tracking info from previous (but not the last) statement.

The change is from Sergei Golubchik
2024-04-10 11:13:46 +10:00
Oleksandr Byelkin
cd28b2479c Merge branch '11.1' into 11.2 2024-04-09 12:12:33 +02:00
Monty
53af3d8c25 Fixed memory leaks in embedded server and mysqltest
This commit fixes the following issues:
- memory leak checking enabled for mysqltest. This cover all cases except
  calls to 'die()' that only happens in case of internal failures in
  mysqltest. die() is not called anymore in the result files differs.
- One can now run mtr --embedded without failures (this crashed or hang
  before)
- cleanup_and_exit() has a new parameter that indicates that it is called
  from die(), in which case we should not do memory leak checks. We now
  always call cleanup_and_exit() instead of exit() to be able to free up
  memory and discover memory leaks.
- Lots of new assert to catch error conditions
- More DBUG statements.
- Fixed that all results are freed in mysqltest (Fixed a memory leak in
  mysqltest when using prepared statements).
- Fixed race condition in do_stmt_close() that caused embedded server
  to not free memory. (Memory leak in mysqltest with embedded server).
- Fixed two memory leaks in embedded server when using prepared statements.
  These memory leaks caused timeout hangs in mtr when server was compiled
  with safemalloc. This issue was not noticed (except as timeouts) as
  memory report checking was done but output of it was disabled.
2024-04-05 12:40:49 +02:00
Marko Mäkelä
683fbced6b Merge 11.0 into 11.1 2024-03-28 12:15:36 +02:00
Marko Mäkelä
fec2fd6add Merge 10.11 into 11.0 2024-03-28 10:51:36 +02:00
Sergei Golubchik
58c7f63176 mysqltest: better error reporting when "shutdown" fails
as a side effect --replace and --error now work for shutdown command
2024-03-27 16:14:56 +01:00
Oleksandr Byelkin
7bcacd767a MDEV-21864 Commands start-all-slaves and stop-all-slaves are not listed in mysqladmin help
Added commands to the help
2024-03-13 14:01:07 +01:00
Marko Mäkelä
015f69a779 MDEV-14448 fixup: clang -Wunused-function 2024-03-11 09:52:59 +02:00
Monty
33dcf8155b Fixed memory leaks in embedded server and mysqltest
This commit fixes the following issues:
- memory leak checking enabled for mysqltest. This cover all cases except
  calls to 'die()' that only happens in case of internal failures in
  mysqltest. die() is not called anymore in the result files differs.
- One can now run mtr --embedded without failures (this crashed or hang
  before)
- cleanup_and_exit() has a new parameter that indicates that it is called
  from die(), in which case we should not do memory leak checks. We now
  always call cleanup_and_exit() instead of exit() to be able to free up
  memory and discover memory leaks.
- Lots of new assert to catch error conditions
- More DBUG statements.
- Fixed that all results are freed in mysqltest (Fixed a memory leak in
  mysqltest when using prepared statements).
- Fixed race condition in do_stmt_close() that caused embedded server
  to not free memory. (Memory leak in mysqltest with embedded server).
- Fixed two memory leaks in embedded server when using prepared statements.
  These memory leaks caused timeout hangs in mtr when server was compiled
  with safemalloc. This issue was not noticed (except as timeouts) as
  memory report checking was done but output of it was disabled.
2024-03-01 10:15:15 +02:00
Monty
86d542887d Fixed memory leaks in embedded server and mysqltest
This commit fixes the following issues:
- memory leak checking enabled for mysqltest. This cover all cases except
  calls to 'die()' that only happens in case of internal failures in
  mysqltest. die() is not called anymore in the result files differs.
- One can now run mtr --embedded without failures (this crashed or hang
  before)
- cleanup_and_exit() has a new parameter that indicates that it is called
  from die(), in which case we should not do memory leak checks. We now
  always call cleanup_and_exit() instead of exit() to be able to free up
  memory and discover memory leaks.
- Lots of new assert to catch error conditions
- More DBUG statements.
- Fixed that all results are freed in mysqltest (Fixed a memory leak in
  mysqltest when using prepared statements).
- Fixed race condition in do_stmt_close() that caused embedded server
  to not free memory. (Memory leak in mysqltest with embedded server).
- Fixed two memory leaks in embedded server when using prepared statements.
  These memory leaks caused timeout hangs in mtr when server was compiled
  with safemalloc. This issue was not noticed (except as timeouts) as
  memory report checking was done but output of it was disabled.
2024-02-29 16:54:49 +02:00
Marko Mäkelä
d73baa402a Merge 10.11 into 11.0 2024-02-20 12:02:01 +02:00
Oleksandr Byelkin
fa69b085b1 Merge branch '11.3' into 11.4 2024-02-15 13:53:21 +01:00
Monty
d6794aa410 Improve error message in mysqltest when sync_with_master fails
In case of failure, the something like the following is now printed:

Slave position:  file: binary.000004  position: 3647
Master position: file: binary.000004  position: 3647
2024-02-14 15:02:21 +01:00
Marko Mäkelä
86c2c89743 Merge 10.6 into 10.11 2024-02-08 15:04:46 +02:00
Marko Mäkelä
466069b184 Merge 10.5 into 10.6 2024-02-08 10:38:53 +02:00
Daniel Black
915d951431 MDEV-4827 mysqldump --dump-slave=2 --master-data=2 doesn't record both
Recording both is useful on a replication relay when the backup
can be used to replace the server, or ack as a new replica to the
server.

If an option=2, commented is selected, allow the alternate option
to exist.

This still disables --dump-slave=1 --master-data=1 as having the
a CHANGE MASTER TO and START SLAVE on different positions would be
confusing and dangerious to the try to execute the output. The
previous behaviour of silently disabling --master-data occurs in
this case.

The commented code related to --dump-slave/--master-data is greatly
expanded for human consumption.

A redundant opt_slave_data= 0 was removed from get_opts. If
--dump-slave=1 or 2, then the only possible value of --master-data
is a valid one.

Re-order to preference gtid based replication.

Based of code from Elena Stepanova.

Review by: Brandon Nesterenko and Anel Husakovic
2024-02-08 15:47:06 +11:00
Sergei Golubchik
6b900330b9 show in mariadb cli whether server cert was verified 2024-02-04 22:19:19 +01:00
Sergei Golubchik
853bdf576f auto-disable --ssl-verify-server-cert in clients, if
* --ssl-verify-server-cert was not enabled explicitly, and
* CA was not specified, and
* fingerprint was not specified, and
* protocol is TCP, and
* no password was provided

insecure passwordless logins are common in test environment, let's
not break them. practically, it hardly makes sense to have strong
MitM protection if an attacker can simply login without a password.

Covers mariadb, mariadb-admin, mariadb-binlog, mariadb-dump
2024-02-04 22:19:19 +01:00
Sergei Golubchik
e951edd80b MDEV-31857 enable --ssl-verify-server-cert by default 2024-02-04 22:19:15 +01:00
Sergei Golubchik
bac0f8999d client support for --ssl-fp and --ssl--fplist
implement --ssl-fp and --ssl-fplist for all clients.
--ssl-fp takes one certificate fingerprint, for example,
00:11:22:33:44:55:66:77:88:99:AA:BB:CC:DD:EE:FF:00:11:22:33

--ssl-fplist takes a path to a file with one fingerprint per line.

if the server's certificate fingerprint matches ssl-fp or is found
in the file - the certificate is considered verified.
If the fingerprint is specified but doesn't match - the connection
is aborted independently from the --ssl-verify-server-cert
2024-02-04 22:19:00 +01:00
Oleksandr Byelkin
d21cb43db1 Merge branch '11.2' into 11.3 2024-02-04 16:42:31 +01:00
Sergei Golubchik
585c096aa5 cleanup: unify client's setting of ssl options 2024-02-03 11:25:13 +01:00
Sergei Golubchik
75bfb4b8a3 deprecate SQL_NOTES variable in favor of NOTE_VERBOSITY
as suggested by Monty
2024-02-03 11:22:20 +01:00
Sergei Golubchik
79580f4f96 Merge branch '11.1' into 11.2 2024-02-02 17:43:57 +01:00
Sergei Golubchik
b6680e0101 Merge branch '11.0' into 11.1 2024-02-02 11:30:47 +01:00
Sergei Golubchik
6ef0b2ee5f Merge branch '10.11' into 11.0 2024-02-01 18:57:08 +01:00
Sergei Golubchik
87e13722a9 Merge branch '10.6' into 10.11 2024-02-01 18:36:14 +01:00
Sergei Golubchik
3f6038bc51 Merge branch '10.5' into 10.6 2024-01-31 18:04:03 +01:00
Sergei Golubchik
01f6abd1d4 Merge branch '10.4' into 10.5 2024-01-31 17:32:53 +01:00
Sergei Golubchik
e5147c8140 regression introduced by MDEV-14448 2024-01-31 15:32:37 +01:00
Oleksandr Byelkin
fe490f85bb Merge branch '10.11' into 11.0 2024-01-30 08:54:10 +01:00
Oleksandr Byelkin
14d930db5d Merge branch '10.6' into 10.11 2024-01-30 08:17:58 +01:00
Oleksandr Byelkin
25c0806867 Merge branch '10.5' into 10.6 2024-01-30 07:43:15 +01:00
Oleksandr Byelkin
50107c4b22 Merge branch '10.4' into 10.5 2024-01-30 07:26:17 +01:00
Vladislav Vaintroub
b0e77c08e5 MDEV-32216 Option --parallel in mariadb-import
This is done for symmetry with mariadb-dump, which does not use threads
but allows parallelism via --parallel

Traditional --use-threads can still be used, it is synonymous
with --parallel
2024-01-29 11:57:53 +01:00
Vladislav Vaintroub
4532dae016 MDEV-32216 option --parallel/-j for mariadb-dump to increase parallelism
At the moment, it only works with --tab, to execute "SELECT INTO OUTFILE"
queries concurrently.

Uses connection_pool for concurrent execution.
2024-01-29 11:57:49 +01:00
Vladislav Vaintroub
ec5db6409d MDEV-32216 Connection pool with asynchronous query execution.
Parallelism is achieved by using mysql_send_query on multiple connections
without waiting for results, and using IO multiplexing (poll/IOCP) to
wait for completions.

Refresh libmariadb to pick up CONC-676 (fixes for IOCP use with named pipe)
2024-01-29 11:54:37 +01:00
Vladislav Vaintroub
9766a834f7 MDEV-32216 preparation - cleanup mysqldump.cc code
- make connect_to_db() return MYSQL*, we'll reuse the function for
  connection pool.

- Remove variable 'mysql_connection', duplicated by variable 'mysql'
- do not attempt to start slave if connection did not succeed,#
  and fix mysqldump.result
2024-01-29 11:54:37 +01:00
Vladislav Vaintroub
a553d55bb6 MDEV-32216 Compile mysqldump as C++ (preparation for using connection pool) 2024-01-29 11:54:37 +01:00
Vladislav Vaintroub
c9c4f15e99 Remove bogus "perl not found" on Windows.
Testing exit code from popen(), comparing it with 1, and deciding that
perl.exe is not there, is a) wrong conclusion, and b) uninteresting,
because MTR always runs with perl, and with MTR_PERL set.

Background:
Recent change in 7af50e4df4 introduced
exit code 1 from perl snippet, that broke Windows CI. Do not want
to debug this ever again.
2024-01-25 14:44:42 +01:00
Vicențiu Ciorbaru
27459b413d
MDEV-14448: Ctrl-C should not exit client
Undo any Windows behaviour changes.
2024-01-23 16:25:58 +02:00
Vicențiu Ciorbaru
3cd8875145
MDEV-14448: Ctrl-C should not exit the client
This patch introduces the following behaviour for Linux while
maintaining old behaviour for Windows:

Ctrl + C (sigint) clears the current buffer and redraws the prompt.
Ctrl-C no longer exits the client if no query is running.
Ctrl-C kills the current running query if there is one. If there is an
error communicating with the server while trying to issue a KILL QUERY,
the client exits. This is in line with the past behaviour of Ctrl-C.
On Linux Ctrl-D can be used to close the client.
On Windows Ctrl-C and Ctrl-BREAK still exits the client if no query is running.
Windows can also exit the client via \q<enter> or exit<enter>.

== Implementation details ==
The Linux implementation has two corner cases, based on which library is
used: libreadline or libedit, both are handled in code to achieve the
same user experience.

Additional code is taken from MySQL, ensuring there is identical
behaviour on Windows, to MySQL's mysql client implementation for other
CTRL- related signals.

* The CTRL_CLOSE, CTRL_LOGOFF, CTRL_SHUTDOWN will issue the equivalent
  of CTRL-C and "end" the program. This ensures that the query is killed
  when the client is closed by closing the terminal, logging off the
  user or shutting down the system. The latter two signals are not sent
  for interactive applications, but it handles the case when a user has
  defined a service to use mysql client to issue a command. See
  https://learn.microsoft.com/en-us/windows/console/handlerroutine

This patch is built on top of the initial work done by Anel Husakovic
<anel@mariadb.org>.
Closes #2815
2024-01-22 14:59:48 +02:00
Marko Mäkelä
35cc4b6c05 Merge 10.11 into 11.0 2024-01-22 10:10:50 +02:00
Marko Mäkelä
b3ca7fa089 Merge 10.6 into 10.11 2024-01-22 08:49:04 +02:00
Sergei Golubchik
3a33ae8601 MDEV-33091 pcre2 headers aren't found on Solaris
use pkg-config to find pcre2, if possible

rename PCRE_INCLUDES to use PKG_CHECK_MODULES naming, PCRE_INCLUDE_DIRS
2024-01-19 20:23:45 +01:00