Commit graph

2887 commits

Author SHA1 Message Date
Marko Mäkelä
52ca2e65af Merge 10.5 into 10.6 2023-11-15 14:10:21 +02:00
Oleksandr Byelkin
9f83a8822f Merge branch '10.5' into mariadb-10.5.23 2023-11-14 08:41:23 +01:00
Oleksandr Byelkin
b83c379420 Merge branch '10.5' into 10.6 2023-11-08 15:57:05 +01:00
Oleksandr Byelkin
6cfd2ba397 Merge branch '10.4' into 10.5 2023-11-08 12:59:00 +01:00
Anel Husakovic
ee77375377 MDEV-26875: Wrong user in SET DEFAULT ROLE error
- Regression introduced in 957cb7b7ba
- Patch 4abb8216a0 change `mysql.user` to `mysql.global_priv` for
`add_anonymous.inc`, update `delete_anonymous.inc`.
- Added test case with `--skip-name-resolve`
- Add test case with anonymous user
  - Disable this test for windows, assignes current user to anonymous
  user.

Reviewed by: <serg@mariadb.com>
2023-10-30 18:39:56 +01:00
Sergei Petrunia
4ed5900626 ANALYZE FORMAT=JSON: Backport block-nl-join.r_unpack_time_ms from 11.0 +fix MDEV-30830.
Also fix it to work with hashed join (MDEV-30830).

Reviewed by: Monty <monty@mariadb.org>
2023-10-27 15:45:00 +02:00
Oleksandr Byelkin
cb4c271355 Fix --view-protocol failures 2023-10-26 13:39:22 +02:00
Alexander Barkov
df72c57d6f MDEV-30048 Prefix keys for CHAR work differently for MyISAM vs InnoDB
Also fixes: MDEV-30050 Inconsistent results of DISTINCT with NOPAD

Problem:

Key segments for CHAR columns where compared using strnncollsp()
for engines MyISAM and Aria.

This did not work correct in case if the engine applyied trailing
space compression.

Fix:

Replacing ha_compare_text() calls to new functions:

- ha_compare_char_varying()
- ha_compare_char_fixed()
- ha_compare_word()
- ha_compare_word_prefix()
- ha_compare_word_or_prefix()

The code branch corresponding to comparison of CHAR column keys
(HA_KEYTYPE_TEXT segment type) now uses ha_compare_char_fixed()
which calls strnncollsp_nchars().

This patch does not change the behavior for the rest of the code:
- comparison of VARCHAR/TEXT column keys
  (HA_KEYTYPE_VARTEXT1, HA_KEYTYPE_VARTEXT2 segments types)
- comparison in the fulltext code
2023-10-24 03:35:48 +04:00
Marko Mäkelä
6991b1c47c Merge 10.5 into 10.6 2023-10-19 13:50:00 +03:00
Monty
185591c1c0 MDEV-31349 test maria.maria-purge failed on 'aria_log.00000002 not found'
The bug was in the test case.
The problem was that maria_empty_logs.inc deleted aria log files before
the server was properly shutdown.
Fixed by waiting for pid file to disappear before starting to delete log
files.

Other things:
- Fixed that translog_purge_at_flush() will not stop deleting files even
  if one file could not be deleted.
2023-10-08 18:16:20 +03:00
Sergei Petrunia
3626379d42 MDEV-20168: main.innodb_icp fails in BB with various wrong execution plans
Add FORCE INDEX and ANALYZE TABLE PERSISTENT FOR ALL to make the plans
stable.
2023-10-03 18:14:18 +03:00
Yuchen Pei
b70d8fbf18
Merge branch '10.5' into 10.6 2023-09-15 12:12:46 +10:00
Yuchen Pei
e95e9a221f
Merge branch '10.4' into 10.5 2023-09-15 12:04:44 +10:00
Anel Husakovic
15cd8542cf MDEV-32004: Cosmetic fixes
- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Anel Husakovic
8d6ae0f2f9 MDEV-32004: Remove extra server_<num>_1 connections during initialization
- Remove extra connections in the form of `server_number_1` for the same server
  during initialization of servers in the `rpl_init.inc` file.
- Remove disconnecting and reconnecting to the same connections,
  since they are not used by the test.
- Update comments about the above.

- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Anel Husakovic
2534e5bc0b MDEV-32004: Parse error in mtr tests when using rpl_check_server_ids parameter
- Fix the calling of the assertion condition when `rpl_check_server_ids` parameter is used.
- Fix comments regarding the default usage and configuration files
extension in this case.

- Reviewer: <knielsen@knielsen-hq.org>
            <brandon.nesterenko@mariadb.com>
2023-09-14 21:35:40 +02:00
Marko Mäkelä
6a470db552 Merge 10.5 into 10.6 2023-09-14 15:25:53 +03:00
Yuchen Pei
cb1965bd9d
Merge branch '10.4' into 10.5 2023-09-14 16:30:11 +10:00
Marko Mäkelä
0dd25f28f7 Merge 10.5 into 10.6 2023-09-11 14:46:39 +03:00
Sergei Golubchik
d890aca6b5 "un-skip" more skipped tests
* version_compile_os can be "linux-systemd", not equal to "Linux"
* main.no-threads forces no-threads scheduler, a check whether it
  has one_thread_per_connection is guaranteed to fail.
2023-09-11 11:23:26 +02:00
Marko Mäkelä
f8f7d9de2c Merge 10.4 into 10.5 2023-09-11 11:29:31 +03:00
Marko Mäkelä
5299f0c45e MDEV-21664 Add opt files for have_innodb_Xk.inc
Currently include/have_innodb_4k.inc etc. files only check that the
server is running with the corresponding page size. I think it would
be more convenient if they actually enforced the setting.
2023-09-11 09:09:02 +03:00
Monty
b08474435f Fix compression tests for s390x
The problem is that s390x is not using the default bzip library we use
on other platforms, which causes compressed string lengths to be differnt
than what mtr tests expects.

Fixed by:
- Added have_normal_bzip.inc, which checks if compress() returns the
  expected length.
- Adjust the results to match the expected one
  - main.func_compress.test & archive.archive
- Don't print lengths that depends on compression library
  - mysqlbinlog compress tests & connect.zip
- Don't print DATA_LENGTH for SET column_compression_zlib_level=1
  - main.column_compression
2023-09-05 12:34:39 +03:00
Marko Mäkelä
2325f8f339 Merge 10.5 into 10.6 2023-08-31 13:01:42 +03:00
Marko Mäkelä
3c86765efe MDEV-23974 fixup: Use standard quotes in have_innodb.inc
This fixes the following test:
set sql_mode=ORACLE;
--source include/have_innodb.inc
2023-08-31 12:08:40 +03:00
Oleksandr Byelkin
6bf8483cac Merge branch '10.5' into 10.6 2023-08-01 15:08:52 +02:00
Oleksandr Byelkin
f291c3df2c Merge branch '10.4' into 10.5 2023-07-27 15:43:21 +02:00
Lena Startseva
9854fb6fa7 MDEV-31003: Second execution for ps-protocol
This patch adds for "--ps-protocol" second execution
of queries "SELECT".
Also in this patch it is added ability to disable/enable
(--disable_ps2_protocol/--enable_ps2_protocol) second
execution for "--ps-prototocol" in testcases.
2023-07-26 17:15:00 +07:00
Sergei Petrunia
6e484c3bd9 MDEV-31577: Make ANALYZE FORMAT=JSON print innodb stats
ANALYZE FORMAT=JSON output now includes table.r_engine_stats which
has the engine statistics. Only non-zero members are printed.

Internally: EXPLAIN data structures Explain_table_acccess and
Explain_update now have handler* handler_for_stats pointer.
It is used to read statistics from handler_for_stats->handler_stats.

The following applies only to 10.9+, backport doesn't use it:

Explain data structures exist after the tables are closed. We avoid
walking invalid pointers using this:
- SQL layer calls Explain_query::notify_tables_are_closed() before
  closing tables.
- After that call, printing of JSON output is disabled. Non-JSON output
  can be printed but we don't access handler_for_stats when doing that.
2023-07-21 16:50:11 +03:00
Oleksandr Byelkin
f52954ef42 Merge commit '10.4' into 10.5 2023-07-20 11:54:52 +02:00
Monty
99bd226059 MDEV-31558 Add InnoDB engine information to the slow query log
The new statistics is enabled by adding the "engine", "innodb" or "full"
option to --log-slow-verbosity

Example output:

 # Pages_accessed: 184  Pages_read: 95  Pages_updated: 0  Old_rows_read: 1
 # Pages_read_time: 17.0204  Engine_time: 248.1297

Page_read_time is time doing physical reads inside a storage engine.
(Writes cannot be tracked as these are usually done in the background).
Engine_time is the time spent inside the storage engine for the full
duration of the read/write/update calls. It uses the same code as
'analyze statement' for calculating the time spent.

The engine statistics is done with a generic interface that should be
easy for any engine to use. It can also easily be extended to provide
even more statistics.

Currently only InnoDB has counters for Pages_% and Undo_% status.
Engine_time works for all engines.

Implementation details:

class ha_handler_stats holds all engine stats.  This class is included
in handler and THD classes.
While a query is running, all statistics is updated in the handler. In
close_thread_tables() the statistics is added to the THD.

handler::handler_stats is a pointer to where statistics should be
collected. This is set to point to handler::active_handler_stats if
stats are requested. If not, it is set to 0.
handler_stats has also an element, 'active' that is 1 if stats are
requested. This is to allow engines to avoid doing any 'if's while
updating the statistics.

Cloned or partition tables have the pointer set to the base table if
status are requested.

There is a small performance impact when using --log-slow-verbosity=engine:
- All engine calls in 'select' will be timed.
- IO calls for InnoDB reads will be timed.
- Incrementation of counters are done on local variables and accesses
  are inline, so these should have very little impact.
- Statistics has to be reset for each statement for the THD and each
  used handler. This is only 40 bytes, which should be neglectable.
- For partition tables we have to loop over all partitions to update
  the handler_status as part of table_init(). Can be optimized in the
  future to only do this is log-slow-verbosity changes. For this to work
  we have to update handler_status for all opened partitions and
  also for all partitions opened in the future.

Other things:
- Added options 'engine' and 'full' to log-slow-verbosity.
- Some of the new files in the test suite comes from Percona server, which
  has similar status information.
- buf_page_optimistic_get(): Do not increment any counter, since we are
  only validating a pointer, not performing any buf_pool.page_hash lookup.
- Added THD argument to save_explain_data_intern().
- Switched arguments for save_explain_.*_data() to have
  always THD first (generates better code as other functions also have THD
  first).
2023-07-07 12:53:18 +03:00
Sergei Golubchik
c05ecda61f fix string literal escaping in views
process multibyte characters correctly, don't escape half of the character
2023-06-02 17:51:40 +02:00
Monty
1ef22e28ad MDEV-26258 Various crashes/asserts/corruptions when Aria encryption is enabled/used, but the encryption plugin is not loaded
The reason for the MDEV reported failures is that the tests are enabling
encryption for Aria but not providing any encryption keys.

Fixed by checking if encryption keys exists before creating the table.

Other things:
- maria.encrypt_wrong-key changed as we now get the error on CREATE
  instead during insert.
2023-05-02 23:37:10 +03:00
Marko Mäkelä
818d5e4814 Merge 10.5 into 10.6 2023-04-25 13:10:33 +03:00
Oleksandr Byelkin
1d74927c58 Merge branch '10.4' into 10.5 2023-04-24 12:43:47 +02:00
Marko Mäkelä
5bada1246d Merge 10.5 into 10.6 2023-04-11 16:15:19 +03:00
Alexander Barkov
ed2adc8c6f MDEV-28190 sql_mode makes MDEV-371 virtual column expressions nondeterministic
This problem was fixed earlier by MDEV-27653.
Adding MTR tests only.
2023-04-06 16:17:50 +04:00
Alexander Barkov
62e137d4d7 Merge remote-tracking branch 'origin/10.4' into 10.5 2023-04-05 16:16:19 +04:00
Sergei Petrunia
0269d82d53 ANALYZE FORMAT=JSON: Backport block-nl-join.r_unpack_time_ms from 11.0 +fix MDEV-30830.
Also fix it to work with hashed join (MDEV-30830).

Reviewed by: Monty <monty@mariadb.org>
2023-04-04 12:18:29 +03:00
Alexander Barkov
8020b1bd73 MDEV-30034 UNIQUE USING HASH accepts duplicate entries for tricky collations
- Adding a new argument "flag" to MY_COLLATION_HANDLER::strnncollsp_nchars()
  and a flag MY_STRNNCOLLSP_NCHARS_EMULATE_TRIMMED_TRAILING_SPACES.
  The flag defines if strnncollsp_nchars() should emulate trailing spaces
  which were possibly trimmed earlier (e.g. in InnoDB CHAR compression).
  This is important for NOPAD collations.

  For example, with this input:
   - str1= 'a '    (Latin letter a followed by one space)
   - str2= 'a  '   (Latin letter a followed by two spaces)
   - nchars= 3
  if the flag is given, strnncollsp_nchars() will virtually restore
  one trailing space to str1 up to nchars (3) characters and compare two
  strings as equal:
  - str1= 'a  '  (one extra trailing space emulated)
  - str2= 'a  '  (as is)

  If the flag is not given, strnncollsp_nchars() does not add trailing
  virtual spaces, so in case of a NOPAD collation, str1 will be compared
  as less than str2 because it is shorter.

- Field_string::cmp_prefix() now passes the new flag.
  Field_varstring::cmp_prefix() and Field_blob::cmp_prefix() do
  not pass the new flag.

- The branch in cmp_whole_field() in storage/innobase/rem/rem0cmp.cc
  (which handles the CHAR data type) now also passed the new flag.

- Fixing UCA collations to respect the new flag.
  Other collations are possibly also affected, however
  I had no success in making an SQL script demonstrating the problem.
  Other collations will be extended to respect this flags in a separate
  patch later.

- Changing the meaning of the last parameter of Field::cmp_prefix()
  from "number of bytes" (internal length)
  to "number of characters" (user visible length).

  The code calling cmp_prefix() from handler.cc was wrong.
  After this change, the call in handler.cc became correct.

  The code calling cmp_prefix() from key_rec_cmp() in key.cc
  was adjusted according to this change.

- Old strnncollsp_nchar() related tests in unittest/strings/strings-t.c
  now pass the new flag.
  A few new tests also were added, without the flag.
2023-04-04 12:30:50 +04:00
Oleksandr Byelkin
ac5a534a4c Merge remote-tracking branch '10.4' into 10.5 2023-03-31 21:32:41 +02:00
Igor Babaev
f33fc2fae5 MDEV-30539 EXPLAIN EXTENDED: no message with queries for DML statements
EXPLAIN EXTENDED for an UPDATE/DELETE/INSERT/REPLACE statement did not
produce the warning containing the text representation of the query
obtained after the optimization phase. Such warning was produced for
SELECT statements, but not for DML statements.
The patch fixes this defect of EXPLAIN EXTENDED for DML statements.
2023-03-25 12:36:59 -07:00
Yuchen Pei
7c91082e39
MDEV-27912 Fixing inconsistency w.r.t. expect files in tests.
mtr uses group suffix, but some existing inc and test files use
server_id for expect files. This patch aims to fix that.

For spider:

With this change we will not have to maintain a separate version of
restart_mysqld.inc for spider, that duplicates code, just because
spider tests use different names for expect files, and shutdown_mysqld
requires magical names for them.

With this change spider tests will also be able to use other features
provided by restart_mysqld.inc without code duplication, like the
parameter $restart_parameters (see e.g. the testcase mdev_29904.test
in commit ef1161e5d4f).

Tests run after this change: default, spider, rocksdb, galera, using
the following command

mtr --parallel=auto --force --max-test-fail=0 --skip-core-file
mtr --suite spider,spider/*,spider/*/* \
    --skip-test="spider/oracle.*|.*/t\..*" --parallel=auto --big-test \
    --force --max-test-fail=0 --skip-core-file
mtr --suite galera --parallel=auto
mtr --suite rocksdb --parallel=auto
2023-03-22 11:55:57 +11:00
Marko Mäkelä
085d0ac238 Merge 10.5 into 10.6 2023-02-28 16:05:21 +02:00
Monty
57c526ffb8 Added detection of memory overwrite with multi_malloc
This patch also fixes some bugs detected by valgrind after this
patch:

- Not enough copy_func elements was allocated by Create_tmp_table() which
  causes an memory overwrite in Create_tmp_table::add_fields()
  I added an ASSERT() to be able to detect this also without valgrind.
  The bug was that TMP_TABLE_PARAM::copy_fields was not correctly set
  when calling create_tmp_table().
- Aria::empty_bits is not allocated if there is no varchar/char/blob
  fields in the table.  Fixed code to take this into account.
  This cannot cause any issues as this is just a memory access
  into other Aria memory and the content of the memory would not be used.
- Aria::last_key_buff was not allocated big enough. This may have caused
  issues with rtrees and ma_extra(HA_EXTRA_REMEMBER_POS) as they
  would use the same memory area.
- Aria and MyISAM didn't take extended key parts into account, which
  caused problems when copying rec_per_key from engine to sql level.
- Mark asan builds with 'asan' in version strihng to detect these in
  not_valgrind_build.inc.
  This is needed to not have main.sp-no-valgrind fail with asan.
2023-02-27 19:25:44 +02:00
Marko Mäkelä
6aec87544c Merge 10.5 into 10.6 2023-02-10 13:03:01 +02:00
Marko Mäkelä
c41c79650a Merge 10.4 into 10.5 2023-02-10 12:02:11 +02:00
Alexander Barkov
0845bce0d9 MDEV-30556 UPPER() returns an empty string for U+0251 in Unicode-5.2.0+ collations for utf8 2023-02-03 18:18:32 +04:00
Oleksandr Byelkin
c3a5cf2b5b Merge branch '10.5' into 10.6 2023-01-31 09:31:42 +01:00
Andrei
dc646c2389 MDEV-30423 Deadlock on Replica during BACKUP STAGE BLOCK_COMMIT on XA transactions
The user XA commit execution branch was caught not have been covered
with MDEV-21953 fixes.

The XA involved deadlock is resolved now to apply the former fixes
pattern.
Along the fixes the following changes have been implemented.
- MDL lock attribute correction
- dissociation of the externally completed XA from the current
  thread's xid_state in the error branches
- cleanup_context() preseves the prepared XA
- wait_for_prior_commit() is relocated to satisfy both
  the binlog ON (log-slave-updates and skip-log-bin)
  and OFF slave execution branches.
2023-01-23 19:01:48 +02:00