Commit graph

448 commits

Author SHA1 Message Date
Oleksandr Byelkin
99b370e023 Merge branch '11.2' into 11.4 2024-05-21 19:38:51 +02:00
Sergei Golubchik
bf5da43e50 Merge branch '11.1' into 11.2 2024-05-13 10:00:26 +02:00
Sergei Golubchik
f0a5412037 Merge branch '11.0' into 11.1 2024-05-13 09:52:30 +02:00
Sergei Golubchik
a6b2f820e0 Merge branch '10.6' into 10.11 2024-05-10 20:02:18 +02:00
Sergei Golubchik
7b53672c63 Merge branch '10.5' into 10.6 2024-05-08 20:06:00 +02:00
Vladislav Vaintroub
029e2a5fd9 MDEV-33876 CMake, zlib - use names compatible with official FindZLIB.cmake
- ZLIB_LIBRARIES, not ZLIB_LIBRARY
- ZLIB_INCLUDE_DIRS, not ZLIB_INCLUDE_DIR

For building libmariadb, ZLIB_LIBRARY/ZLIB_INCLUDE_DIR are still defined
This workaround will be removed later.
2024-05-03 21:48:47 +02:00
Oleksandr Byelkin
fa69b085b1 Merge branch '11.3' into 11.4 2024-02-15 13:53:21 +01:00
Oleksandr Byelkin
d21cb43db1 Merge branch '11.2' into 11.3 2024-02-04 16:42:31 +01:00
Sergei Golubchik
79580f4f96 Merge branch '11.1' into 11.2 2024-02-02 17:43:57 +01:00
Sergei Golubchik
b6680e0101 Merge branch '11.0' into 11.1 2024-02-02 11:30:47 +01:00
Kristian Nielsen
d039346a7a MDEV-4991: GTID binlog indexing
Improve the performance of slave connect using B+-Tree indexes on each binlog
file. The index allows fast lookup of a GTID position to the corresponding
offset in the binlog file, as well as lookup of a position to find the
corresponding GTID position.

This eliminates a costly sequential scan of the starting binlog file
to find the GTID starting position when a slave connects. This is
especially costly if the binlog file is not cached in memory (IO
cost), or if it is encrypted or a lot of slaves connect simultaneously
(CPU cost).

The size of the index files is generally less than 1% of the binlog data, so
not expected to be an issue.

Most of the work writing the index is done as a background task, in
the binlog background thread. This minimises the performance impact on
transaction commit. A simple global mutex is used to protect index
reads and (background) index writes; this is fine as slave connect is
a relatively infrequent operation.

Here are the user-visible options and status variables. The feature is on by
default and is expected to need no tuning or configuration for most users.

binlog_gtid_index
  On by default. Can be used to disable the indexes for testing purposes.

binlog_gtid_index_page_size (default 4096)
  Page size to use for the binlog GTID index. This is the size of the nodes
  in the B+-tree used internally in the index. A very small page-size (64 is
  the minimum) will be less efficient, but can be used to stress the
  BTree-code during testing.

binlog_gtid_index_span_min (default 65536)
  Control sparseness of the binlog GTID index. If set to N, at most one
  index record will be added for every N bytes of binlog file written.
  This can be used to reduce the number of records in the index, at
  the cost only of having to scan a few more events in the binlog file
  before finding the target position

Two status variables are available to monitor the use of the GTID indexes:

  Binlog_gtid_index_hit
  Binlog_gtid_index_miss

The "hit" status increments for each successful lookup in a GTID index.
The "miss" increments when a lookup is not possible. This indicates that the
index file is missing (eg. binlog written by old server version
without GTID index support), or corrupt.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2024-01-27 12:09:54 +01:00
Marko Mäkelä
b3ca7fa089 Merge 10.6 into 10.11 2024-01-22 08:49:04 +02:00
Sergei Golubchik
3a33ae8601 MDEV-33091 pcre2 headers aren't found on Solaris
use pkg-config to find pcre2, if possible

rename PCRE_INCLUDES to use PKG_CHECK_MODULES naming, PCRE_INCLUDE_DIRS
2024-01-19 20:23:45 +01:00
Oleksandr Byelkin
34272bd6a5 Merge branch '11.2' into 11.3 2023-11-14 18:33:03 +01:00
Oleksandr Byelkin
0427c4739e MariaDB 11.1.3 release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEF39AEP5WyjM2MAMF8WVvJMdM0dgFAmVSLiUACgkQ8WVvJMdM
 0dhj4A/7B2GIx75Mv4IcExE2s4bfR7sOKZzvjWbMHysovMHhsHAV5fHN7dRQojyV
 HxmSY8lxykm/LMoJ8RASmrojRZsgvkJ84z+fLK7is327Vms7fW7ZWc3eqotIgs7I
 m9Dz3+wiexvl6NKeHnafTZtkJOe8MEqZEGPV1e8V4I3SAJQWyQLnRr4si/VmjAMi
 miKuieTuKoZUYSkdNLwicEFHXysgg6/U8367sgMsJe9V3HYSD3pVQJ/nboTL5uZL
 vTbmEPS1pICKPvPu75DdedSdxSASMyXis9/IWtk13NqPPzX16uHtjkhffAuBT3+k
 CUgRggTYAuoF3MjvyspIS3pdC/73PBb1O+w/9vlHPiwSXVn3d48Ay55uvFgM/pVB
 UKLorw+As0oH2N1HWUp/d4Rbvrnjdq5OgzhmMTrWDAtYyrNU9Jw5S1CAp+G/s2dD
 5j+FUPBBnHo5UfxI+EVTqUggm56R+vJTx4H3q82n05bdJTJYNJ+nixvsYuf7hS3f
 oEqJAUizgGI3h6FGPD9bN0HSYGblEeNgAYv1YogfVX/Eq10RriWic9PtxxOxgOmE
 n+UhdH4YTTyaTv0jssWTJVmVNzjjXMvI4aB8A1FkXeIz2iohSziSkJzaBuzNq2QY
 kKHr8XqiyNnckcoRxfoxNPtrWcmiykpHOBFnuyMRWoXPKcr7idc=
 =ShdC
 -----END PGP SIGNATURE-----

Merge tag '11.1' into 11.2

MariaDB 11.1.3 release
2023-11-14 18:28:37 +01:00
Oleksandr Byelkin
0f5613a25f Merge branch '11.0' into 11.1 2023-11-08 18:03:08 +01:00
Marko Mäkelä
5a8fca5a4f Merge 10.6 into 10.10 2023-10-23 18:43:36 +03:00
Sergei Petrunia
4941ac9192 MDEV-32113: utf8mb3_key_col=utf8mb4_value cannot be used for ref
(Variant#3: Allow cross-charset comparisons, use a special
CHARSET_INFO to create lookup keys. Review input addressed.)

Equalities that compare utf8mb{3,4}_general_ci strings, like:

  WHERE ... utf8mb3_key_col=utf8mb4_value    (MB3-4-CMP)

can now be used to construct ref[const] access and also participate
in multiple-equalities.
This means that utf8mb3_key_col can be used for key-lookups when
compared with an utf8mb4 constant, field or expression using '=' or
'<=>' comparison operators.

This is controlled by optimizer_switch='cset_narrowing=on', which is
OFF by default.

IMPLEMENTATION
Item value comparison in (MB3-4-CMP) is done using utf8mb4_general_ci.
This is valid as any utf8mb3 value is also an utf8mb4 value.

When making index lookup value for utf8mb3_key_col, we do "Charset
Narrowing": characters that are in the Basic Multilingual Plane (=BMP) are
copied as-is, as they can be represented in utf8mb3. Characters that are
outside the BMP cannot be represented in utf8mb3 and are replaced
with U+FFFD, the "Replacement Character".

In utf8mb4_general_ci, the Replacement Character compares as equal to any
character that's not in BMP. Because of this, the constructed lookup value
will find all index records that would be considered equal by the original
condition (MB3-4-CMP).

Approved-by: Monty <monty@mariadb.org>
2023-10-19 17:24:30 +03:00
Sergei Petrunia
e987b9350c MDEV-31496: Make optimizer handle UCASE(varchar_col)=...
(Review input addressed)
(Added handling of UPDATE/DELETE and partitioning w/o index)

If the properties of the used collation allow, do the following
equivalent rewrites:

1. UPPER(key_col)=expr  ->  key_col=expr
   expr=UPPER(key_col)  ->  expr=key_col
   (also rewrite both sides of the equality at the same time)

2. UPPER(key_col) IN (constant-list)  -> key_col IN (constant-list)

- Mark utf8mb{3,4}_general_ci as collations that allow this.
- Add optimizer_switch='sargable_casefold=ON' to control this.
  (ON by default in this patch)
- Cover the rewrite in Optimizer Trace, rewrite name is
  "sargable_casefold_removal".
2023-09-12 17:14:43 +03:00
Dmitry Shulga
9e48460bdb MDEV-5816: Stored programs: validation of stored program statements
This is the prerequisite patch to move the sp_instr class and classes derived
from it into the files sp_instr.cc/sp_instr.h. The classes sp_lex_cursor and
sp_lex_keeper are also moved to the files files sp_instr.cc/sp_instr.h.

Additionally,
  * all occurrences of macroses NULL, FALSE, TRUE are replaced
    with the corresponding C++ keywords nullptr, false, true.
  * the keyword 'override' is added in and the keyword 'virtual' is removed
    from signatures of every virtual method implemented in classes derived
    from the base class sp_instr.
  * the keyword 'final' is added into declaration of the class sp_lex_keeper
    since this class shouldn't have a derived class by design.
  * the function cmp_rqp_locations is made static since it is not called
    outside the file sp_instr.cc.
  * the function subst_spvars() is moved into the file sp_instr.cc since this
    function used only by the method sp_instr_stmt::execute
2023-07-20 17:46:44 +07:00
Alexander Barkov
75f25e4ca7 MDEV-30164 System variable for default collations
This patch adds a way to override default collations
(or "character set collations") for desired character sets.

The SQL standard says:
> Each collation known in an SQL-environment is applicable to one
> or more character sets, and for each character set, one or more
> collations are applicable to it, one of which is associated with
> it as its character set collation.

In MariaDB, character set collations has been hard-coded so far,
e.g. utf8mb4_general_ci has been a hard-coded character set collation
for utf8mb4.

This patch allows to override (globally per server, or per session)
character set collations, so for example, uca1400_ai_ci can be set as a
character set collation for Unicode character sets
(instead of compiled xxx_general_ci).

The array of overridden character set collations is stored in a new
(session and global) system variable @@character_set_collations and
can be set as a comma separated list of charset=collation pairs, e.g.:

SET @@character_set_collations='utf8mb3=uca1400_ai_ci,utf8mb4=uca1400_ai_ci';

The variable is empty by default, which mean use the hard-coded
character set collations (e.g. utf8mb4_general_ci for utf8mb4).

The variable can also be set globally by passing to the server startup command
line, and/or in my.cnf.
2023-07-17 14:56:17 +04:00
Rucha Deodhar
358b8495f5 MDEV-27128: Implement JSON Schema Validation FUNCTION
Implementation:
Implementation is made according to json schema validation draft 2020

JSON schema basically has same structure as that of json object, consisting
of key-value pairs. So it can be parsed in the same manner as
any json object.

However, none of the keywords are mandatory, so making guess about the
json value type based only on the keywords would be incorrect.
Hence we need separate objects denoting each keyword.

So during create_object_and_handle_keyword() we create appropriate objects
based on the keywords and validate each of them individually on the json
document by calling respective validate() function if the type matches.
If any of them fails, return false, else return true.
2023-04-26 11:00:08 +05:30
Oleg Smirnov
f0b665f880 MDEV-8320 Allow index usage for DATE(col) <=> const and YEAR <=> const
Rewrite datetime comparison conditions into sargeable. For example,
    YEAR(col) <= val  ->  col <= YEAR_END(val)
    YEAR(col) <  val  ->  col <  YEAR_START(val)
    YEAR(col) >= val  ->  col >= YEAR_START(val)
    YEAR(col) >  val  ->  col >  YEAR_END(val)
    YEAR(col) =  val  ->  col BETWEEN YEAR_START(val) AND YEAR_END(val)
Do the same with DATE(col), for example:
    DATE(col) <= val  ->  col <= DAY_END(val)

After such a rewrite index lookup on column "col" can be employed
2023-04-25 20:21:35 +07:00
Oleksandr Byelkin
d86ad1f127 Merge branch '10.8' into 10.9 2022-10-17 12:39:25 +02:00
Oleksandr Byelkin
f3fddc1b4a Merge branch '10.7' into 10.8 2022-10-17 08:44:12 +02:00
Oleksandr Byelkin
ec2b30e736 Merge branch '10.6' into 10.7 2022-10-16 21:40:33 +02:00
Oleksandr Byelkin
822694bd56 Merge branch '10.5' into 10.6 2022-10-15 23:47:33 +02:00
Marko Mäkelä
66e44afd94 Merge 10.4 into 10.5 2022-10-13 17:05:30 +03:00
Marko Mäkelä
f404911557 Merge 10.3 into 10.4 2022-10-13 16:50:26 +03:00
Marko Mäkelä
1f5615360c Silence clang 13 -Wunused-but-set-variable
WITH_EMBEDDED_SERVER compiles the SQL parsers separately.
Thanks to Vladislav Vaintroub for helping with this.

Fixes up commit e05ab0cfc5
2022-10-13 14:43:35 +03:00
Marko Mäkelä
f53f64b7b9 Merge 10.8 into 10.9 2022-07-28 10:47:33 +03:00
Marko Mäkelä
f79cebb4d0 Merge 10.7 into 10.8 2022-07-28 10:33:26 +03:00
Marko Mäkelä
742e1c727f Merge 10.6 into 10.7 2022-07-27 18:26:21 +03:00
Vladislav Vaintroub
494a75c851 MDEV-28888 : Embedded MariaDB does not build on Windows
Add limited support for building embedded library (DLL only).
2022-07-03 17:11:56 +02:00
Alexander Barkov
0c4c064f98 MDEV-27743 Remove Lex::charset
This patch also fixes:

MDEV-27690 Crash on `CHARACTER SET csname COLLATE DEFAULT` in column definition
MDEV-27853 Wrong data type on column `COLLATE DEFAULT` and table `COLLATE some_non_default_collation`
MDEV-28067 Multiple conflicting column COLLATE clauses are not rejected
MDEV-28118 Wrong collation of `CAST(.. AS CHAR COLLATE DEFAULT)`
MDEV-28119 Wrong column collation on MODIFY + CONVERT
2022-03-22 17:12:15 +04:00
Sergei Petrunia
00377dbae8 Fix embedded to work 2022-01-19 18:10:10 +03:00
Vladislav Vaintroub
7acf31783c Add new option NOT_EMBEDDED, for plugins
Means, plugin will not be available in embedded, even if compiled-in
2021-11-09 16:57:27 +02:00
Vladislav Vaintroub
284ed64336 Fix broken build dependency, when compiling without perfschema
libfmt must be built before sql or sql_embedded target, since compilation
of item_strfunc.cc depends on it.

Previously, GenServerSource depended on libfmt,yet this GenServerSource
is not a  direct or indirect dependency of sql, if -DPLUGIN_PERFSCHEMA=NO
is used in cmake configuration step.
2021-10-12 13:28:10 +02:00
Sergei Golubchik
fe65ca0176 don't build bundled libfmt, we use it in header-only mode anyway 2021-10-12 13:28:10 +02:00
Alan Cueva
e214e60201 MDEV-25015 Custom formatting of strings in MariaDB queries
SFORMAT() SQL function that uses fmtlib (https://fmt.dev/)
for python-like (also Rust, C++20, etc) string formatting

Only fmtlib 7.0.0+ is supported, older fmtlib
produces different results in the test.

No native support for temporal and decimal values,
* TIME_RESULT is handled as STRING_RESULT
* DECIMAL_RESULT as REAL_RESULT
2021-10-12 13:27:36 +02:00
Marko Mäkelä
b36d6f92a8 Merge 10.6 into 10.7 2021-09-30 11:01:07 +03:00
Marko Mäkelä
a49e394399 Merge 10.5 into 10.6 2021-09-30 10:38:44 +03:00
Vladislav Vaintroub
e5a9dcfda2 MDEV-23306 Fix build dependency 2021-09-28 22:26:05 +02:00
Monty
8d08971c84 Removed CREATE/DROP TABLESPACE and related commands
- DISCARD/IMPORT TABLESPACE are the only tablespace commands left
- TABLESPACE arguments for CREATE TABLE and ALTER ... ADD PARTITION are
  ignored.
- Tablespace names are not shown anymore in .frm and not shown in
  information schema

Other things
- Removed end spaces from sql/CMakeList.txt
2021-09-14 18:04:09 +03:00
Marko Mäkelä
a722ee88f3 Merge 10.5 into 10.6 2021-06-01 11:39:38 +03:00
Sergei Golubchik
4777097fee followup: rename generated files to have distinct names 2021-05-27 00:40:23 +02:00
Monty
496a14e187 Move debug_crash_here to it's own source files 2021-05-19 22:54:13 +02:00
Monty
02b6cef45e Move all ddl log code to ddl_log.cc and ddl_log.h
Part of prepration for: MDEV-17567 Atomic DDL

No notable code changes except moving code around
2021-05-19 22:54:11 +02:00
Marko Mäkelä
916b237b3f Merge 10.5 into 10.6 2021-05-07 15:00:27 +03:00
Nikita Malyavin
3f55c56951 Merge branch bb-10.4-release into bb-10.5-release 2021-05-05 23:57:11 +03:00