Commit graph

200261 commits

Author SHA1 Message Date
Sergei Golubchik
71d92723bf MDEV-33696 main.dyncol and ctype_unicode_casefold_bmp.inc in --view
temporarily disable view protocol
2024-03-21 11:37:12 +01:00
Sergei Golubchik
f10805fc29 MDEV-31531 fix wsrep tests after 929c2e06aa 2024-03-21 11:32:03 +01:00
Sergei Golubchik
d0c47cd161 wsrep.wsrep_provider_plugin_defaults update to 26.4.17 2024-03-21 11:32:03 +01:00
Sergei Golubchik
473ee856d1 MDEV-28152 fix UBSAN error
sql_sequence.h:233:19: runtime error: signed integer overflow: -9223372036854775808 + -1 cannot be represented in type 'long long int'

followup for 374783c3d9
2024-03-21 11:32:03 +01:00
Sergei Golubchik
2be3b8e8a3 main.information_schema-big is neither big nor not_embedded 2024-03-21 11:32:03 +01:00
Vladislav Vaintroub
14ec1536ac MDEV-33519 Remove WITH_SSL=<custom_location_of_openssl> option
This commit removes the WITH_SSL=<custom_location_of_openssl> option,
leaving only -DWITH_SSL=bundled/system.

The rationale behind this removal is as follows:

1. The WITH_SSL=<custom_location_of_openssl> option is obscure
and not widely used.

2. There is no added value in this option compared to using
OPENSSL_ROOT_DIR. In fact, the availability of "helpful" MySQL options
might discourage users from exploring proper CMake options independently.

3. Users may incorrectly assume full MySQL compatibility even with this
option, including undocumented behaviors such as MySQL's preference for static libraries
with WITH_SSL=<custom_location_of_openssl>.

This change simplifies the configuration options and encourages users to
adopt more standardized and documented practices.
2024-03-18 13:30:03 +01:00
Alexander Barkov
1e889a6e6c MDEV-33621 Unify duplicate code in my_wildcmp_uca_impl() and my_wildcmp_unicode_impl()
This is a refactoring patch, it does not change the behaviour.
The MTR tests are being added only to cover the LIKE predicate better.
(these tests should have been added earlier under terms of MDEV 9711).
This patch does not need its own specific MTR tests.

Moving the duplicate code into a new shared file ctype-wildcmp.inl
and including it from multiple places, to define the following functions:

- my_wildcmp_uca_impl(), in ctype-uca.c

  For utf8mb3, utf8mb4, ucs2, utf16, utf32, using cs->cset->mb_wc().
  For UCA based collations.

- my_wildcmp_mb2_or_mb4_general_ci_impl(), in ctype-ucs2.c:

  For ucs2, utf16, utf32, using cs->cset->mb_wc().
  For general_ci-style collations:
      - xxx_general_ci
      - xxx_general_mysql500_ci
      - xxx_general_nopad_ci

- my_wildcmp_mb2_or_mb4_bin_impl(), in ctype-ucs2.c:

  For ucs2, utf16, utf32, using cs->cset->mb_wc().
  For _bin collations:
      - xxx_bin
      - xxx_nopad_bin

- my_wildcmp_utf8mb3_general_ci_impl(), in ctype-utf8.c

  Optimized for utf8mb3, using my_mb_wc_utf8mb3_quick().

  For general_ci-style collations:
      - utf8mb3_general_ci
      - utf8mb3_general_mysql500_ci
      - utf8mb3_general_nopad_ci

- my_wildcmp_utf8mb4_general_ci_impl(), in ctype-utf8.c

  Optimized for utf8mb4, using my_mb_wc_utf8mb4_quick().

  For general_ci-style collations:
      - utf8mb4_general_ci
      - utf8mb4_general_nopad_ci
2024-03-12 09:33:20 +04:00
Alexander Barkov
9e7afa7782 Extra tests for MDEV-30716 Wrong casefolding in xxx_unicode_520_ci for U+0700..U+07FF
New tests display additional information about characters from the BMP range:

- A summary with a COUNT(*) for all distinct combinations of properties
  telling how the "=" and the "LIKE" predicates compare characters to their
  LOWER() and UPPER() variants.

- A detailed list of trciky characters
  for which the "=" and the "LIKE" predicates compare
  LOWER(c)/UPPER(c) variants as not equal to just "c".

Tricky characters include:
 - Turkish letters: ı - small dotless letter i
 - Croatian letters: precombined contractions for Dž, Dz, Lj, Nj
 - Units of measurement: Ω,K,Å (Ohm, Kelvin, Angstrom)
   These ones look very similar to Greek letter Omega,
   Latin letter Kra, Swedish/Finnish letter A with a ring above.
2024-03-07 17:38:36 +04:00
Alexander Barkov
929c2e06aa MDEV-31531 Remove my_casedn_str() and my_caseup_str()
Under terms of MDEV 27490 we'll add support for non-BMP identifiers
and upgrade casefolding information to Unicode version 14.0.0.
In Unicode-14.0.0 conversion to lower and upper cases can increase octet length
of the string, so conversion won't be possible in-place any more.

This patch removes virtual functions performing in-place casefolding:
  - my_charset_handler_st::casedn_str()
  - my_charset_handler_st::caseup_str()
and fixes the code to use the non-inplace functions instead:
  - my_charset_handler_st::casedn()
  - my_charset_handler_st::caseup()
2024-02-28 22:20:29 +04:00
Vladislav Vaintroub
de9c357284 MDEV-32841 Add tests for new counters.
Only add innodb_async_reads/writes_total_count here.
For all other counters, writing reproducible tests is rather tricky.
2024-02-28 13:31:19 +01:00
Vladislav Vaintroub
59df9f26bb MDEV-32841 - review fixes 2024-02-28 13:31:19 +01:00
Vladislav Vaintroub
f8bb99bf4a MDEV-32841 Use chrono::duration<double> instead of chrono::duration<float>
Requested by Marko during review
2024-02-28 13:31:19 +01:00
Vladislav Vaintroub
01466adc13 MDEV-32841 Provide Innodb async IO statistics
Provide some statistics about asynchronous IO reads and writes:
 - number of pending operations
 - number of completion callbacks that are currently being executed
 - number of completion callbacks that are currently queued
   (due to restriction on number of IO threads)
 - total number of IOs finished
 - total time to wait for free IO slot
 - total number of completions that were queued.

Also revert tpool InnoDB perfschema instrumentation (MDEV-31048)
That instrumentation of cache mutex did not bring any revelation (
the mutex is taken for a couple of instructions), and made it impossible
to use tpool outside of the server (e.g in mariadbimport/dump)
2024-02-28 13:31:19 +01:00
Yuchen Pei
374783c3d9
MDEV-28152 Features for sequences
- Add `as <int_type>` to sequence creation options
  - int_type can be signed or unsigned integer types, including
    tinyint, smallint, mediumint, int and bigint
  - Limitation: when alter sequence as <new_int_type>, cannot have any
    other alter options in the same statement
  - Limitation: increment remains signed longlong, and the hidden
    constraint (cache_size x abs(increment) < longlong_max) stays for
    unsigned types. This means for bigint unsigned, neither
    abs(increment) nor (cache_size x abs(increment)) can be between
    longlong_max and ulonglong_max
- Truncating maxvalue and minvalue from user input to the nearest max
  or min value of the type, plus or minus 1. When the truncation
  happens, a warning is emitted
- Information schema table for sequences
2024-02-21 11:19:27 +11:00
Sergei Golubchik
eeba940311 remove deprecated since 10.4 2024-02-17 17:10:25 +01:00
Sergei Golubchik
04f0504831 11.5 branch 2024-02-17 17:10:25 +01:00
Daniel Bartholomew
9b6e267bfd bump the VERSION 2024-02-17 15:30:50 +01:00
Oleksandr Byelkin
fa69b085b1 Merge branch '11.3' into 11.4 2024-02-15 13:53:21 +01:00
Sergei Golubchik
3ae6680eec update 32bit rdiffs 2024-02-15 00:03:27 +01:00
Yuchen Pei
068a6819eb MDEV-33441 Do not deinit plugin variables when retry requested
After MDEV-31400, plugins are allowed to ask for retries when failing
initialisation. However, such failures also cause plugin system
variables to be deleted (plugin_variables_deinit()) before retrying
and are not re-added during retry.

We fix this by checking that if the plugin has requested a retry the
variables are not deleted. Because plugin_deinitialize() also calls
plugin_variables_deinit(), if the retry fails, the variables will
still be deleted.

Alternatives considered:

- remove the plugin_variables_deinit() from plugin_initialize() error
handling altogether. We decide to take a more conservative approach
here.

- re-add the system variables during retry. It is more complicated
than simply iterating over plugin->system_vars and call
my_hash_insert(). For example we will need to assign values to
the test_load field and extract more code from test_plugin_options(),
if that is possible.
2024-02-14 18:45:03 +01:00
Sergei Golubchik
fe07ac31b1 MDEV-31857 fix galera.MW-284
followup for abcd23add2
2024-02-14 16:19:22 +01:00
Sergei Golubchik
8dee23cdee MDEV-31857 fix galera.galera_var_notify_ssl_ipv6
it was able to enable --ssl-verify-server-cert if explicily requested,
now it can also disable it, if explicitly requested.
2024-02-14 16:13:02 +01:00
Monty
d6794aa410 Improve error message in mysqltest when sync_with_master fails
In case of failure, the something like the following is now printed:

Slave position:  file: binary.000004  position: 3647
Master position: file: binary.000004  position: 3647
2024-02-14 15:02:21 +01:00
Monty
18dfcfdecf MDEV-31404 Implement binlog_space_limit
binlog_space_limit is a variable in Percona server used to limit the total
size of all binary logs.

This implementation is based on code from Percona server 5.7.

In MariaDB we decided to call the variable max-binlog-total-size to be
similar to max-binlog-size. This makes it easier to find in the output
from 'mariadbd --help --verbose'). MariaDB will also support
binlog_space_limit for compatibility with Percona.

Some internal notes to explain implementation notes:

- When running MariaDB does not delete binary logs that are either
  used by slaves or have active xid that are not yet committed.

Some implementation notes:

- max-binlog-total-size is by default 0 (no limit).
- max-binlog-total-size can be changed without server restart.
- Binlog file sizes are checked on startup, or if
  max-binlog-total-size is set to a value > 0, not for every log write.
  The total size of all binary logs is cached and dynamically updated
  when updating the binary log on binary log rotation.
- max-binlog-total-size is checked against existing log files during
  serverstart, binlog rotation, FLUSH LOGS, when writing to binary log
  or when max-binlog-total-size changes value.
- Option --slave-connections-needed-for-purge with 1 as default added.
  This allows one to ensure that we do not delete binary logs if there
  is less than 'slave-connections-needed-for-purge' connected.
  Without this option max-binlog-total-size would potentially delete
  binlogs needed by slaves on server startup or when a slave disconnects
  as there are then no connected slaves to protect active binlogs.
- PURGE BINARY LOGS TO ... will be executed as if
  slave-connectitons-needed-for-purge would be zero. In other words
  it will do the purge even if there is no slaves connected. If there
  are connected slaves working on the logs, these will be protected.
- If binary log is on and max-binlog-total_size <> 0 then the status
  variable 'Binlog_disk_use' shows the current size of all old binary
  logs + the state of the current one.
- Removed test of strcmp(log_file_name, log_info.log_file_name) in
  purge_logs_before_date() as this is tested in can_purge_logs()
- To avoid expensive calls of log_in_use() we cache the result for the
  last log that is in use by a slave. Future calls to can_purge_logs()
  for this binary log will be quickly detected and false will be returned
  until a slave starts working on a new log.
- Note that after a binary log rotation caused by max_binlog_size,
  the last log will not be purged directly as it is still in use
  internally. The next binary log write will purge binlogs if needed.

Reviewer:Kristian Nielsen <knielsen@knielsen-hq.org>
2024-02-14 15:02:21 +01:00
Sergei Golubchik
9933a8cc88 update C/C 2024-02-14 15:02:21 +01:00
Sergei Golubchik
2c445b5962 fix debian dependencies for mariadb-test
libnet-ssleay-perl wasn't auto-detected
2024-02-14 15:00:20 +01:00
Sergei Golubchik
eeb5cbaece MDEV-33326 Port Spider/ODBC from ES
Revert "Deb: Stop shipping mariadb-plugin-spider separately, include in server"

This reverts commit 9945d482af.
2024-02-14 14:59:16 +01:00
Otto Kekäläinen
9945d482af Deb: Stop shipping mariadb-plugin-spider separately, include in server
Instead of having a separate plugin, simply include Spider in the main
MariaDB Server package and let users manually enable on server installs
where they want to run it.
2024-02-13 15:25:03 +00:00
Nikita Malyavin
22e41dae88 MDEV-32501 KEY_PERIOD_USAGE reveals information to unprivileged user
Restrict access to KEY_PERIOD_USAGE: show the constraint record iff any
non-select privilege on any table column is granted.

Also drop the unprivileged user in the end of test and add merge anchor.
2024-02-12 22:26:06 +01:00
Nikita Malyavin
5c2f8c017c MDEV-32503 Queries from I_S.KEY_PERIOD_USAGE do not obey case-sensitivity
Set the correct values for idx_field1, idx_field2 (i.e. db and table name
fields) of ST_SCHEMA_TABLE instance.
2024-02-12 22:26:06 +01:00
Nikita Malyavin
261993f568 add period to buildbot_suites.bat 2024-02-12 22:26:06 +01:00
Nikita Malyavin
d4b5f7a503 MDEV-32205 crash in get_schema_key_period_usage_record without InnoDB
Move table open result processing to the caller

* st_schema_table::process_table doesn't have to check whether the table
was opened successfully
* It also doesn't have to check for a thd error and convert it to a warning
* This simplifies adding new tables into information_schema
* A callback still can output some info to a user in case of error. In
order to do this, I_S_EXTENDED_ERROR_HANDLING should be specified in
i_s_requested_object.
2024-02-12 22:26:06 +01:00
Nikita Malyavin
62d35a074f Add Statement::sql_command_flags() function.
Now one can call thd->sql_command_flags() instead of
sql_command_flags[thd->lex->sql_command].
2024-02-12 22:26:06 +01:00
Nikita Malyavin
4246c0fa01 MDEV-22597 Add views for periods in information_schema
Two new information_schema views are added:
* PERIOD table -- columns TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME,
  PERIOD_NAME, START_COLUMN_NAME, END_COLUMN_NAME.
* KEY_PERIOD_USAGE -- works similar to KEY_COLUMN_USAGE, but for periods.
  Columns CONSTRAINT_CATALOG, CONSTRAINT_SCHEMA, CONSTRAINT_NAME,
  TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME, PERIOD_NAME

Two new columns are added to the COLUMNS view:
 IS_SYSTEM_TIME_PERIOD_START, IS_SYSTEM_TIME_PERIOD_END - contain YES/NO.
2024-02-12 22:26:06 +01:00
Nikita Malyavin
85f9df29c0 sql_show: reduce ifdefs around grants checks
Embedded server has an option to opt out grant checks.
sql_show.cc: reduce ifdefs by using embedded acl stub calls
sql_acl.cc: add stub for embedded get_column_grant
2024-02-12 22:26:06 +01:00
Nikita Malyavin
ba1c5914aa fix get_schema_privileges_for_show 2024-02-12 22:26:06 +01:00
Nikita Malyavin
16ad26a367 sql_show: extract get_schema_privileges_for_show 2024-02-12 22:26:06 +01:00
Nikita Malyavin
eb2481280f sql_show: Fix narrowing conversion from the get_column_grant return result. 2024-02-12 22:26:06 +01:00
Nikita Malyavin
8a882827ff cleanup: add store_yesno() for fields that can only take "YES"/"NO" values 2024-02-12 22:26:05 +01:00
Daniel Black
711b8671be pr template: ask about release notes 2024-02-12 14:58:25 +00:00
Vladislav Vaintroub
4eac842c8f MDEV-33430 - Fix self-signed certificate errors on Windows
Adjust test after fixing the C/C.

On Windows, use --host=127.0.0.2 to fake "insecure" transport
with TCP connection for test purposes. 127.0.0.2 is loopback address,
that can be used instead of usual 127.0.0.1

Unfortunately, this technique does not work on all *nixes the same,
notably neither on BSDs nor Solaris. Thus default --host=localhost
remains "insecure" transport,when TCP is used. but it is not that critical,
the "self-signed" is not nearly as annoying on *nixes as it is on Windows.
2024-02-09 14:57:54 +01:00
Alexander Barkov
9500575f0a MDEV-33428 Error messages ER_PACKAGE_ROUTINE_* are not good enough
Changing the format in error messages:
- ER_PACKAGE_ROUTINE_IN_SPEC_NOT_DEFINED_IN_BODY
- ER_PACKAGE_ROUTINE_FORWARD_DECLARATION_NOT_DEFINED

from
  "Subroutine 'db.pkg.f1' ..."

to a more clear:
  "FUNCTION `db.pkg.f1` ..."
  "PROCEDURE `db.pkg.p1` ..."
2024-02-08 16:01:20 +04:00
Brandon Nesterenko
9b1ea69049 Revert "MDEV-7850: Extend GTID Binlog Events with Thread Id"
This reverts commit c37b2087b4.

In c37b20887, when re-binlogging a GTID event on a replica,
it will overwrite the thread_id from the primary to be the
value of the slave applier (SQL thread or parallel worker).

This should be the value of the original thread_id on the
master connection though, to both help track temporary
tables, and be consistent with Query_log_event.

Reverting the commit to re-target 11.5, so we can re-test
with the corrected thread_id.
2024-02-05 05:56:53 -07:00
Alexander Barkov
e30e9fc628 MDEV-33386 Wrong error message on GRANT .. ON PACKAGE no_such_package ..
When displaying the ER_SP_DOES_NOT_EXIST error, use
Sp_handler::type_lex_cstring() to the the underlying
object type:
- PROCEDURE
- FUNCTION
- PACKAGE
- PACKAGE BODY
instead of hard-coded "FUNCTION or PROCEDURE".
2024-02-05 14:41:44 +04:00
Sergei Golubchik
2e83ab4126 MDEV-32473 --disable-ssl doesn't disable it 2024-02-04 22:19:19 +01:00
Sergei Golubchik
6b900330b9 show in mariadb cli whether server cert was verified 2024-02-04 22:19:19 +01:00
Sergei Golubchik
853bdf576f auto-disable --ssl-verify-server-cert in clients, if
* --ssl-verify-server-cert was not enabled explicitly, and
* CA was not specified, and
* fingerprint was not specified, and
* protocol is TCP, and
* no password was provided

insecure passwordless logins are common in test environment, let's
not break them. practically, it hardly makes sense to have strong
MitM protection if an attacker can simply login without a password.

Covers mariadb, mariadb-admin, mariadb-binlog, mariadb-dump
2024-02-04 22:19:19 +01:00
Sergei Golubchik
abcd23add2 MDEV-31857 enable --ssl-verify-server-cert by default in the internal client
enable ssl + ssl_verify_server_cert in the internal client too

* fix replication tests to disable master_ssl_verify_server_cert
  because accounts are passwordless - except rpl.rpl_ssl1
* fix federated/federatedx/connect to disable SSL_VERIFY_SERVER_CERT
  because they cannot configure an ssl connection
* fix spider to disable ssl_verify_server_cert, if configuration
  says so, as spider _can_ configure an ssl connection
* memory leak in embedded test-connect
2024-02-04 22:19:19 +01:00
Sergei Golubchik
e0c30390a7 MDEV-31855 validate ssl certificates using client password in the internal client
port the client-side implementation from C/C to the internal client.
add the test.
2024-02-04 22:19:19 +01:00
Sergei Golubchik
386df8793b disable SSL via named pipes in the internal client
because it doesn't work. CONC-635.
2024-02-04 22:19:19 +01:00