Commit graph

200263 commits

Author SHA1 Message Date
Sergei Golubchik
e0c30390a7 MDEV-31855 validate ssl certificates using client password in the internal client
port the client-side implementation from C/C to the internal client.
add the test.
2024-02-04 22:19:19 +01:00
Sergei Golubchik
386df8793b disable SSL via named pipes in the internal client
because it doesn't work. CONC-635.
2024-02-04 22:19:19 +01:00
Sergei Golubchik
3c36ed18ba free mysql->connector_fd correctly in the internal client
it's not an ssl option, so shouldn't be in mysql_ssl_free(),
which frees ssl options, and only unless CLIENT_REMEMBER_OPTIONS is set.

mysql->connector_fd must be freed when mysql->net.vio is closed
and fd becomes no longer valid
2024-02-04 22:19:19 +01:00
Sergei Golubchik
2f13f7d78f change how self-signed certs are accepted by internal client
use SSL_VERIFY_PEER with the "always ok" callback,
instead of SSL_VERIFY_NONE with no callback.

The latter doesn't work correctly in wolfSSL, it accepts self-signed
certificates just fine (as in OpenSSL), but after that
SSL_get_verify_result() returns X509_V_OK, while it returns an error
(e.g. X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN) in OpenSSL.
2024-02-04 22:19:19 +01:00
Sergei Golubchik
05a421eb36 cleanup: X509_check_host() in the internal client
X509_check_host() and X509_check_ip_asc() exist in all
supported SSL libraries

in OpenSSL >= 1.0.2 and in the bundled WolfSSL

And X509_free() handles NULL pointers all right.
2024-02-04 22:19:19 +01:00
Sergei Golubchik
f4e174e113 cleanup: ssl handling in the internal rpl client
* type of mi->ssl_verify_server_cert must be my_bool, because it's
  passed by address to mysql_options(), and the latter expects my_bool
* explicitly disable ssl in MYSQL if mi->ssl is 0
* remove dead code (`#ifdef NOT_USED`)
* remove useless casts and checks replacing empty strings with NULL
  (new_VioSSLFd() does that internally)
2024-02-04 22:19:19 +01:00
Sergei Golubchik
e951edd80b MDEV-31857 enable --ssl-verify-server-cert by default 2024-02-04 22:19:15 +01:00
Sergei Golubchik
ea921fd836 enable --ssl in the server by default
except in bootstrap
2024-02-04 22:19:00 +01:00
Sergei Golubchik
9f93630ded MDEV-31856 use ephemeral ssl certificates
if the server is started with --ssl but without neither --ssl-key nor
--ssl-cert, let it automatically generate a self-signed certificate.
It's generated in memory only and never saved to disk.
2024-02-04 22:19:00 +01:00
Sergei Golubchik
d33a8ab107 wrong error for bare --ssl on the server side
when neither --ssl-key nor --ssl-cert were set, the errror
was "Private key does not match the certificate public key"

changed to "Unable to get certificate"
2024-02-04 22:19:00 +01:00
Sergei Golubchik
d772c4fb04 cleanup 2024-02-04 22:19:00 +01:00
Sergei Golubchik
68f0af2bf1 test SSL MitM attack
verify that --ssl-verify-server-cert detects cert mismatch,
but with --disable-ssl-verify-server-cert the connection succeeds
2024-02-04 22:19:00 +01:00
Sergei Golubchik
bac0f8999d client support for --ssl-fp and --ssl--fplist
implement --ssl-fp and --ssl-fplist for all clients.
--ssl-fp takes one certificate fingerprint, for example,
00:11:22:33:44:55:66:77:88:99:AA:BB:CC:DD:EE:FF:00:11:22:33

--ssl-fplist takes a path to a file with one fingerprint per line.

if the server's certificate fingerprint matches ssl-fp or is found
in the file - the certificate is considered verified.
If the fingerprint is specified but doesn't match - the connection
is aborted independently from the --ssl-verify-server-cert
2024-02-04 22:19:00 +01:00
Sergei Golubchik
1ef1bab99e MDEV-31855 validate ssl certificates using client password
if the client enabled --ssl-verify-server-cert, then
the server certificate is verified as follows:

* if --ssl-ca or --ssl-capath were specified, the cert must have
  a proper signature by the specified CA (or CA in the path)
  and the cert's hostname must match the server's hostname.
  If the cert isn't signed or a hostname is wrong - the
  connection is aborted.

* if MARIADB_OPT_TLS_PEER_FP was used and the fingerprint matches,
  the connection is allowed, if it doesn't match - aborted.

* If the connection uses unix socket or named pipes - it's allowed.
  (consistent with server's --require-secure-transport behavior)

otherwise the cert is still in doubt, we don't know if we can trust
it or there's an active MitM in progress.

* If the user has provided no password or the server requested an
  authentication plugin that sends the password in cleartext -
  the connection is aborted.

* Perform the authentication. If the server accepts the password,
  it'll send SHA2(scramble || password hash || cert fingerprint)
  with the OK packet.

* Verify the SHA2 digest, if it matches - the connection is allowed,
  otherwise it's aborted.
2024-02-04 22:18:55 +01:00
Oleksandr Byelkin
d21cb43db1 Merge branch '11.2' into 11.3 2024-02-04 16:42:31 +01:00
Sergei Golubchik
585c096aa5 cleanup: unify client's setting of ssl options 2024-02-03 11:25:13 +01:00
Sergei Golubchik
03094bbc8a cleanup: octet2hex takes an uchar* argument
char is a character, uchar is an octet.
casts removed (or added) as needed
2024-02-03 11:22:21 +01:00
Sergei Golubchik
d7699c51eb test.cnf files should !include default_my.cnf
not default_mysqld.cnf. The latter has only server settings,
it misses mtr-specific client configuration

Except for spider, that doesn't use mysqld.1 server
and default_my.cnf starts it automatically.
Spider tests have to include both default_mysqld.cnf and
default_client.cnf
2024-02-03 11:22:20 +01:00
Sergei Golubchik
ec5403ffb3 clarify CR_OK_HANDSHAKE_COMPLETE
it's for client auth plugins only, server auth plugin should never
return it, because they cannot send a correct OK packet.
(OK packet is quite complex and carries a lot of information that
only the server knows)
2024-02-03 11:22:20 +01:00
Sergei Golubchik
75bfb4b8a3 deprecate SQL_NOTES variable in favor of NOTE_VERBOSITY
as suggested by Monty
2024-02-03 11:22:20 +01:00
Sergei Golubchik
22da0de6d9 gitignore scripts/mariadb_sys_schema.sql 2024-02-03 11:22:20 +01:00
Sergei Golubchik
79580f4f96 Merge branch '11.1' into 11.2 2024-02-02 17:43:57 +01:00
Vladislav Vaintroub
2f5174e556 MDEV-33075 Resolve server shutdown issues on macOS, Solaris, and FreeBSD
This commit addresses multiple server shutdown problems observed on macOS,
Solaris, and FreeBSD:

1. Corrected a non-portable assumption where socket shutdown was expected
to wake up poll() with listening sockets in the main thread.

Use more robust self-pipe to wake up poll() by writing to the pipe's write
end.

2. Fixed a random crash on macOS from pthread_kill(signal_handler)
when the signal_handler was detached and the thread had already exited.

Use more robust `kill(getpid(), SIGTERM)` to wake up the signal handler
thread.

3. Made sure, that signal handler thread always exits once `abort_loop` is
set, and also calls `my_thread_end()` and clears `signal_thread_in_use`
when exiting.

This fixes warning "1 thread did not exit"  by `my_global_thread_end()`
seen on FreeBSD/macOS when the process is terminated via signal.

Additionally, the shutdown code underwent light refactoring
for better readability and maintainability:
- Modified `break_connect_loop()` to no longer wait for the main thread,
  aligning behavior with Windows (since 10.4).
- Removed dead code related to the unused `USE_ONE_SIGNAL_HAND`
  preprocessor constant.
- Eliminated support for `#ifndef HAVE_POLL` in `handle_connection_sockets`
  This code is also dead, since 10.4
2024-02-02 16:18:00 +01:00
Sergei Golubchik
b6680e0101 Merge branch '11.0' into 11.1 2024-02-02 11:30:47 +01:00
Sergei Golubchik
6ef0b2ee5f Merge branch '10.11' into 11.0 2024-02-01 18:57:08 +01:00
Sergei Golubchik
87e13722a9 Merge branch '10.6' into 10.11 2024-02-01 18:36:14 +01:00
Sergei Golubchik
b5c367cd88 MDEV-32815 test main.func_sformat Locale + test failures under Fedora 39 (fmt-10.0.0+)
FMT_STATIC_THOUSANDS_SEPARATOR stopped working in 10.0.0
Let's not use this fmt version for now
2024-02-01 18:35:18 +01:00
Sergei Golubchik
15c75ad083 pcre.cmake: always check the library with check_library_exists()
even if pkg-config has it. otherwise build dependencies
aren't detected.
2024-02-01 11:26:36 +01:00
Brandon Nesterenko
dd95c58b58 MDEV-33331: IO Thread Relay Log Inconsistent Statistics After MDEV-32551
After MDEV-32551, in a master/slave setup, if the replica's IO thread
quickly and successively reconnects (i.e quickly running
STOP SLAVE IO_THREAD followed by START SLAVE IO_THREAD), the relay log
rotation behavior changes. That is, MDEV-32551 changed the logic of the
binlog_dump_thread on the primary, such that it can stop itself before
sending any events if it sees a new connection has been created to a
replica with the same server_id. Pre MDEV-32551, the connection would
establish and it would send a "fake" rotate event to populate the
log name. Post MDEV-32551, the connection stops itself, and a rotate
event is not sent.

This made the test rpl.rpl_mariadb_slave_capability unstable because
it is reliant on the name of the relay logs (which is dependent on the
number of rotates); and the pre-amble of the test would quickly
start/stop the IO thread. There a binlog dump thread could end itself
before sending a rotate event to the replica, thereby changing the name
of the relay log.

This patch fixes this by adding in a synchronization in-between IO thread
restarts, such that it waits for the primary's binlog dump threads to
sync up with the state of the replica.
2024-01-31 22:18:31 +01:00
Sergei Golubchik
2278f3503e fix columnstore compilation on fc39 2024-01-31 22:02:59 +01:00
Sergei Golubchik
3f6038bc51 Merge branch '10.5' into 10.6 2024-01-31 18:04:03 +01:00
Sergei Golubchik
01f6abd1d4 Merge branch '10.4' into 10.5 2024-01-31 17:32:53 +01:00
Sergei Golubchik
46e3a7658b funcs_1.innodb_views times out in --ps 2024-01-31 17:07:46 +01:00
Sergei Golubchik
e5147c8140 regression introduced by MDEV-14448 2024-01-31 15:32:37 +01:00
Sergei Golubchik
d1744ee7a2 MDEV-33343 spider.mdev_28739_simple fails in buildbot
test disabled, until fixed
2024-01-31 15:32:37 +01:00
Oleksandr Byelkin
908c9cf90c workaround for MDEV-33218 2024-01-30 17:00:15 +01:00
Oleksandr Byelkin
fe490f85bb Merge branch '10.11' into 11.0 2024-01-30 08:54:10 +01:00
Oleksandr Byelkin
14d930db5d Merge branch '10.6' into 10.11 2024-01-30 08:17:58 +01:00
Oleksandr Byelkin
25c0806867 Merge branch '10.5' into 10.6 2024-01-30 07:43:15 +01:00
Oleksandr Byelkin
50107c4b22 Merge branch '10.4' into 10.5 2024-01-30 07:26:17 +01:00
Brandon Nesterenko
c75905cacb MDEV-33327: rpl_seconds_behind_master_spike Sensitive to IO Thread Stop Position
rpl.rpl_seconds_behind_master_spike uses the DEBUG_SYNC mechanism to
count how many format descriptor events (FDEs) have been executed,
to attempt to pause on a specific relay log FDE after executing
transactions. However, depending on when the IO thread is stopped,
it can send an extra FDE before sending the transactions, forcing
the test to pause before executing any transactions, resulting in a
table not existing, that is attempted to be read for COUNT.

This patch fixes this by no longer counting FDEs, but rather by
programmatically waiting until the SQL thread has executed the
transaction and then automatically activating the DEBUG_SYNC point
to trigger at the next relay log FDE.
2024-01-30 06:58:44 +01:00
Vladislav Vaintroub
b0e77c08e5 MDEV-32216 Option --parallel in mariadb-import
This is done for symmetry with mariadb-dump, which does not use threads
but allows parallelism via --parallel

Traditional --use-threads can still be used, it is synonymous
with --parallel
2024-01-29 11:57:53 +01:00
Vladislav Vaintroub
a5802ed51e MDEV-32216 add tests for mariadb-dump --parallel
-  --parallel=N with or without --single-transaction
-  Error cases (too many connections, emulate error on one connection)
-  Windows specific test for named pipe connections
2024-01-29 11:57:53 +01:00
Vladislav Vaintroub
4532dae016 MDEV-32216 option --parallel/-j for mariadb-dump to increase parallelism
At the moment, it only works with --tab, to execute "SELECT INTO OUTFILE"
queries concurrently.

Uses connection_pool for concurrent execution.
2024-01-29 11:57:49 +01:00
Vladislav Vaintroub
ec5db6409d MDEV-32216 Connection pool with asynchronous query execution.
Parallelism is achieved by using mysql_send_query on multiple connections
without waiting for results, and using IO multiplexing (poll/IOCP) to
wait for completions.

Refresh libmariadb to pick up CONC-676 (fixes for IOCP use with named pipe)
2024-01-29 11:54:37 +01:00
Vladislav Vaintroub
9766a834f7 MDEV-32216 preparation - cleanup mysqldump.cc code
- make connect_to_db() return MYSQL*, we'll reuse the function for
  connection pool.

- Remove variable 'mysql_connection', duplicated by variable 'mysql'
- do not attempt to start slave if connection did not succeed,#
  and fix mysqldump.result
2024-01-29 11:54:37 +01:00
Vladislav Vaintroub
a553d55bb6 MDEV-32216 Compile mysqldump as C++ (preparation for using connection pool) 2024-01-29 11:54:37 +01:00
Monty
4dbf55bbfc Disable perfschema.misc_session_status for 32 bit
32bit uses less memory so the test for max_memory_usage does not work
2024-01-27 16:51:45 +02:00
Monty
ed76a2e8ac Updated some 32 bit result files in sys_vars 2024-01-27 16:51:15 +02:00
Monty
e20693c167 Fixed some wrong printf() usage after changing m_table_id to ulonglong
This caused some crashes on 32 bit platforms.
2024-01-27 16:29:40 +02:00