Commit graph

1225 commits

Author SHA1 Message Date
Yuchen Pei
d0f8dfbcf0
Merge branch '11.1' into 11.2 2023-10-27 18:11:56 +11:00
Marko Mäkelä
9b2a65e41a Merge 11.0 into 11.1 2023-10-19 08:26:16 +03:00
Marko Mäkelä
be24e75229 Merge 10.11 into 11.0 2023-10-19 08:12:16 +03:00
Marko Mäkelä
2ecc0443ec Merge 10.10 into 10.11 2023-10-17 16:04:21 +03:00
Marko Mäkelä
d5e15424d8 Merge 10.6 into 10.10
The MDEV-29693 conflict resolution is from Monty, as well as is
a bug fix where ANALYZE TABLE wrongly built histograms for
single-column PRIMARY KEY.
Also includes a fix for safe_malloc error reporting.

Other things:
- Copied main.log_slow from 10.4 to avoid mtr issue

Disabled test:
- spider/bugfix.mdev_27239 because we started to get
  +Error	1429 Unable to connect to foreign data source: localhost
  -Error	1158 Got an error reading communication packets
- main.delayed
  - Bug#54332 Deadlock with two connections doing LOCK TABLE+INSERT DELAYED
    This part is disabled for now as it fails randomly with different
    warnings/errors (no corruption).
2023-10-14 13:36:11 +03:00
Monty
d4347177c7 Change SEL_ARG::MAX_SEL_ARGS to a user defined variable optimizer_max_sel_args
This allows a user to to change the default value of MAX_SEL_ARGS (16000)
in the rare case where they neeed more generated SEL_ARGS (as part of
the range optimizer)
2023-10-03 08:25:31 +03:00
Monty
4e9322e2ff MDEV-32203 Raise notes when an index cannot be used on data type mismatch
Raise notes if indexes cannot be used:
- in case of data type or collation mismatch (diferent error messages).
- in case if a table field was replaced to something else
  (e.g. Item_func_conv_charset) during a condition rewrite.

Added option to write warnings and notes to the slow query log for
slow queries.

New variables added/changed:

- note_verbosity, with is a set of the following options:
  basic            - All old notes
  unusable_keys    - Print warnings about keys that cannot be used
                     for select, delete or update.
  explain          - Print unusable_keys warnings for EXPLAIN querys.

The default is 'basic,explain'. This means that for old installations
the only notable new behavior is that one will get notes about
unusable keys when one does an EXPLAIN for a query. One can turn all
of all notes by either setting note_verbosity to "" or setting sql_notes=0.

- log_slow_verbosity has a new option 'warnings'. If this is set
  then warnings and notes generated are printed in the slow query log
  (up to log_slow_max_warnings times per statement).

- log_slow_max_warnings   - Max number of warnings written to
                            slow query log.

Other things:
- One can now use =ALL for any 'set' variable to set all options at once.
  For example using "note_verbosity=ALL" in a config file or
  "SET @@note_verbosity=ALL' in SQL.
- mysqldump will in the future use @@note_verbosity=""' instead of
  @sql_notes=0 to disable notes.
- Added "enum class Data_type_compatibility" and changing the return type
  of all Field::can_optimize*() methods from "bool" to this new data type.

Reviewer & Co-author: Alexander Barkov <bar@mariadb.com>
- The code that prints out the notes comes mainly from Alexander
2023-10-03 08:25:31 +03:00
Sergei Golubchik
37e854f34a Merge branch '11.1' into 11.2 2023-09-29 16:01:59 +02:00
Sergei Golubchik
05d850d4b3 Merge branch '11.0' into 11.1 2023-09-29 13:58:47 +02:00
Sergei Golubchik
3f6bccb888 Merge branch '10.11' into 11.0 2023-09-29 12:24:54 +02:00
Marko Mäkelä
6a470db552 Merge 10.5 into 10.6 2023-09-14 15:25:53 +03:00
Yuchen Pei
cb1965bd9d
Merge branch '10.4' into 10.5 2023-09-14 16:30:11 +10:00
Daniel Black
1831f8e4d7 MDEV-31369 Disable TLS v1.0 and 1.1 for MariaDB
Remove TLSv1.1 from the default tls_version system variable.

Output a warning if TLSv1.0 or TLSv1.1 are selected.

Thanks Tingyao Nian for the feature request.
2023-09-13 20:17:29 +10:00
Monty
69c420be3d Added support for --skip-secure-file-priv
This works the same as secure-file-priv="", but is more obvious way to
turn of secure-file-priv.
2023-09-09 15:14:44 +03:00
Sergei Golubchik
18ddde4826 Merge branch '11.1' into 11.2 2023-08-18 00:59:16 +02:00
Nikita Malyavin
8aa1a9e6a7 MDEV-31812 Add switch to old_mode to disable non-locking ALTER
Add LOCK_ALTER_TABE_COPY bit to old_mode. Disables online copy by default,
but still allows to force it with explicit lock=none
2023-08-15 14:00:28 +02:00
Oleksandr Byelkin
f5fae75652 Merge branch '11.0' into 11.1 2023-08-09 08:25:14 +02:00
Oleksandr Byelkin
51f9d62005 Merge branch '10.11' into 11.0 2023-08-09 07:53:48 +02:00
Oleksandr Byelkin
036df5f970 Merge branch '10.10' into 10.11 2023-08-08 14:57:31 +02:00
Oleksandr Byelkin
ced243a099 Merge branch '10.9' into 10.10 2023-08-05 20:34:09 +02:00
Oleksandr Byelkin
34a8e78581 Merge branch '10.6' into 10.9 2023-08-04 08:01:06 +02:00
Sergei Golubchik
5de39c5ae3 MDEV-9069 extend AES_ENCRYPT() and AES_DECRYPT() to support IV and the algorithm
AES_ENCRYPT(str, key, [, iv [, mode ]])
AES_DECRYPT(str, key, [, iv [, mode ]])

mode is aes-{128,192,256}-{ecb,cbc,ctr} e.g. "aes-128-cbc".

and a @@block_encryption_mode variable for the default value of mode

change in behavior: AES_ENCRYPT(str, key) can no longer
be used in persistent virtual columns (and alike)
2023-08-02 13:29:48 +02:00
Sergei Golubchik
75f5cc478f MDEV-30905 Remove old_alter_table variable 2023-08-02 13:29:48 +02:00
Oleksandr Byelkin
6bf8483cac Merge branch '10.5' into 10.6 2023-08-01 15:08:52 +02:00
Oleksandr Byelkin
4235c133ae Merge branch '10.4' into 10.5 2023-07-31 10:14:46 +02:00
Kristian Nielsen
d632c85bb7 MDEV-31723: Crash on SET SESSION gtid_seq_no= DEFAULT
A simple "SET SESSION gtid_seq_no= DEFAULT" did not work, it would straight
up crash the server! Also, explicitly setting gtid_seq_no to 0 gave an error
in --gtid-strict-mode=1.

Setting to DEFAULT or 0 should disable any prior setting of
gtid_seq_no, so that the next transaction is allocated the next GTID
in sequence, as normal.

Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
2023-07-30 22:00:43 +02:00
Marko Mäkelä
e81fa34502 Merge 11.1 into 11.2 2023-07-26 15:49:24 +03:00
Marko Mäkelä
c6ac1e39b6 Merge 11.0 into 11.1 2023-07-26 15:13:43 +03:00
Marko Mäkelä
f2b4972bd4 Merge 10.11 into 11.0 2023-07-26 15:13:06 +03:00
Marko Mäkelä
bce3ee704f Merge 10.10 into 10.11 2023-07-26 14:44:43 +03:00
Marko Mäkelä
b1b47264d2 Merge 10.9 into 10.10 2023-07-26 14:17:36 +03:00
Alexander Barkov
75f25e4ca7 MDEV-30164 System variable for default collations
This patch adds a way to override default collations
(or "character set collations") for desired character sets.

The SQL standard says:
> Each collation known in an SQL-environment is applicable to one
> or more character sets, and for each character set, one or more
> collations are applicable to it, one of which is associated with
> it as its character set collation.

In MariaDB, character set collations has been hard-coded so far,
e.g. utf8mb4_general_ci has been a hard-coded character set collation
for utf8mb4.

This patch allows to override (globally per server, or per session)
character set collations, so for example, uca1400_ai_ci can be set as a
character set collation for Unicode character sets
(instead of compiled xxx_general_ci).

The array of overridden character set collations is stored in a new
(session and global) system variable @@character_set_collations and
can be set as a comma separated list of charset=collation pairs, e.g.:

SET @@character_set_collations='utf8mb3=uca1400_ai_ci,utf8mb4=uca1400_ai_ci';

The variable is empty by default, which mean use the hard-coded
character set collations (e.g. utf8mb4_general_ci for utf8mb4).

The variable can also be set globally by passing to the server startup command
line, and/or in my.cnf.
2023-07-17 14:56:17 +04:00
Marko Mäkelä
7cde5c539b Merge 10.6 into 10.9 2023-07-10 11:22:21 +03:00
Monty
99bd226059 MDEV-31558 Add InnoDB engine information to the slow query log
The new statistics is enabled by adding the "engine", "innodb" or "full"
option to --log-slow-verbosity

Example output:

 # Pages_accessed: 184  Pages_read: 95  Pages_updated: 0  Old_rows_read: 1
 # Pages_read_time: 17.0204  Engine_time: 248.1297

Page_read_time is time doing physical reads inside a storage engine.
(Writes cannot be tracked as these are usually done in the background).
Engine_time is the time spent inside the storage engine for the full
duration of the read/write/update calls. It uses the same code as
'analyze statement' for calculating the time spent.

The engine statistics is done with a generic interface that should be
easy for any engine to use. It can also easily be extended to provide
even more statistics.

Currently only InnoDB has counters for Pages_% and Undo_% status.
Engine_time works for all engines.

Implementation details:

class ha_handler_stats holds all engine stats.  This class is included
in handler and THD classes.
While a query is running, all statistics is updated in the handler. In
close_thread_tables() the statistics is added to the THD.

handler::handler_stats is a pointer to where statistics should be
collected. This is set to point to handler::active_handler_stats if
stats are requested. If not, it is set to 0.
handler_stats has also an element, 'active' that is 1 if stats are
requested. This is to allow engines to avoid doing any 'if's while
updating the statistics.

Cloned or partition tables have the pointer set to the base table if
status are requested.

There is a small performance impact when using --log-slow-verbosity=engine:
- All engine calls in 'select' will be timed.
- IO calls for InnoDB reads will be timed.
- Incrementation of counters are done on local variables and accesses
  are inline, so these should have very little impact.
- Statistics has to be reset for each statement for the THD and each
  used handler. This is only 40 bytes, which should be neglectable.
- For partition tables we have to loop over all partitions to update
  the handler_status as part of table_init(). Can be optimized in the
  future to only do this is log-slow-verbosity changes. For this to work
  we have to update handler_status for all opened partitions and
  also for all partitions opened in the future.

Other things:
- Added options 'engine' and 'full' to log-slow-verbosity.
- Some of the new files in the test suite comes from Percona server, which
  has similar status information.
- buf_page_optimistic_get(): Do not increment any counter, since we are
  only validating a pointer, not performing any buf_pool.page_hash lookup.
- Added THD argument to save_explain_data_intern().
- Switched arguments for save_explain_.*_data() to have
  always THD first (generates better code as other functions also have THD
  first).
2023-07-07 12:53:18 +03:00
Marko Mäkelä
b7ee3c7b9c Merge 11.1 into 11.2 2023-07-04 08:22:18 +03:00
Sergei Golubchik
279b0db8c6 MDEV-29827 collateral cleanup
* removed universal suppression of everything "Event Schedule" everywhere
* added suppressions in tests as needed
* moved events test to the events suite
* renamed -master.opt -> .opt
* added standard test header
* verified in the test that the error, indeed, was written into the log
* removed useless suppressions
* removed ER_EVENTS_NO_ACL, replaced with ER_OPTION_PREVENTS_STATEMENT
* fixed error message to say exactly what option disabled event scheduler
  instead of "this or that or that, you figure it out"
* also fixed old message for SET event_scheduler=
  (it was also non-translatable)
* changed to use sql_print_error() when an error is not sent to the user
* removed duplicate hard-coded error message
2023-07-02 12:15:11 +02:00
Zhibo Zhang
34bbf37f4f MDEV-30188: Ensure all binlog* variables are visible as system variables
Turn the remaining three `binlog*` options binlog_do_db, binlog_ignore_db,
binlog_rows_event_max_size into global variables so that they can be
visible from the SQL user level. This is for audit / secure
configuration check purposes.

Create new MTR tests to make sure that the newly created global
variables can be visible from the command line interface.

Behavior before the code change:

    MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE
        -> Variable_name LIKE 'binlog_do_db' OR
        -> Variable_name LIKE 'binlog_ignore_db' OR
        -> Variable_name LIKE 'binlog_row_event_max_size';
    Empty set (0.001 sec)

Behavior after the code change:

    MariaDB [(none)]> SHOW GLOBAL VARIABLES WHERE
        -> Variable_name LIKE 'binlog_do_db' OR
        -> Variable_name LIKE 'binlog_ignore_db' OR
        -> Variable_name LIKE 'binlog_row_event_max_size';
    +---------------------------+-------+
    | Variable_name             | Value |
    +---------------------------+-------+
    | binlog_do_db              |       |
    | binlog_ignore_db          |       |
    | binlog_row_event_max_size | 8192  |
    +---------------------------+-------+
    3 rows in set (0.001 sec)

Note:

For `binlog_do_db` and `binlog_ignore_db`, we add a new class
`Sys_var_binlog_filter` to handle the dynamically-composable command line
options for `binlog_do_db` and `binlog_ignore_db`. Below
is the motivation:

When the users start the server with the option
    --binlog-do-db="database1" --binlog-do-db="database2"
The expected behavior is that the system should allow replication for
both `database1` and `database2`, which is the logic of the original
code.

However, when turning the variables into system variables, the
functionality does not exist any more, since system variables will only
handle the last occurrence of the option, and in this case, the system
will only be able to handle `database2`.

Copyright:

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services, Inc.
2023-06-19 09:38:19 +01:00
Sergei Golubchik
cbabb95915 Merge branch '11.0' into 11.1 2023-06-05 20:15:15 +02:00
Oleksandr Byelkin
06d03dcdd3 Merge branch '10.10' into 10.11 2023-05-03 21:05:34 +02:00
Monty
3bdc5542dc MDEV-30812: Improve output cardinality estimates for hash join
Introduces @@optimizer_switch flag: hash_join_cardinality

When this option is on, use EITS statistics to produce tighter bounds
for hash join output cardinality.

This patch is an extension / replacement to a similar patch in 10.6

New features:
- optimizer_switch hash_join_cardinality is on by default
- records_out is set to fanout when HASH is used
- Fixed bug in is_eits_usable: The function did not work with views
2023-05-03 21:44:57 +03:00
Oleksandr Byelkin
13a294a2c9 Merge branch '10.9' into 10.10 2023-05-03 14:09:13 +02:00
Oleksandr Byelkin
cf56f2d7e8 Merge branch '10.8' into 10.9 2023-05-03 13:27:59 +02:00
Oleksandr Byelkin
f0f1f2de0e Merge branch '10.6' into 10.8 2023-05-03 11:33:57 +02:00
Sergei Petrunia
4329ec5d3b MDEV-30812: Improve output cardinality estimates for hash join
Introduce @@optimizer_switch flag: hash_join_cardinality

When it is on, use EITS statistics to produce tighter bounds for
hash join output cardinality.

Amended by Monty.

Reviewed by: Monty <monty@mariadb.org>
2023-04-28 16:24:08 +03:00
Marko Mäkelä
aa6ba99310 Merge 10.11 into 11.0 2023-04-27 15:11:18 +03:00
Marko Mäkelä
7b17e0d4ac Merge 10.10 into 10.11 2023-04-27 14:03:24 +03:00
Marko Mäkelä
4f16e95310 Merge 10.9 into 10.10 2023-04-27 13:36:36 +03:00
Marko Mäkelä
436d8ef6b5 Merge 10.8 into 10.9 2023-04-27 10:03:26 +03:00
Marko Mäkelä
bbd261bb2e Merge 10.6 into 10.8 2023-04-27 09:53:56 +03:00
Marko Mäkelä
bb1d1dc846 Merge 10.5 into 10.6 2023-04-27 09:48:27 +03:00