Commit graph

194957 commits

Author SHA1 Message Date
Marko Mäkelä
fae0ccad6e Merge 10.5 into 10.6 2022-04-21 17:46:40 +03:00
Marko Mäkelä
620c55e708 Merge 10.4 into 10.5 2022-04-21 15:33:50 +03:00
Vlad Lesin
1b558dd462 MDEV-27919 mariabackup --log-copy-interval is measured in millisecondss in 10.5 and in microseconds in 10.6
Multiply polling interval by 1000.
2022-04-21 15:24:59 +03:00
Marko Mäkelä
fdec842fd7 MDEV-28371 Assertion fold == id.fold() failed in buf_flush_check_neighbor()
Due to 32-bit arithmetics, SRV_TMP_SPACE_ID page number 0x200002 would be
folded to 0, which is incompatible with the assumption that was made in
commit 7cffb5f6e8 (MDEV-23399).

page_id_t::fold(): Compute in the native word width instead of uint32_t.
On 64-bit platforms, an alternative would be to return the 64-bit m_id
directly, but that was measured to cause a performance regression.

fil_space_t::open(): Invoke fil_node_t::find_metadata() when the
tablespace is being created. In this way, we will actually detect
that the temporary tablespace resides on SSD. (During database
creation, also the system tablespace will correctly be detected as
residing on SSD.)
2022-04-21 14:09:23 +03:00
Daniel Black
f0c52bfe3b MDEV-28263: mariadb-tzinfo-to-sql (postfix)
I missed include/no_protocol.inc already existed.
2022-04-21 19:13:58 +10:00
Daniel Black
c47069ead7 MDEV-28313 InnoDB transactions are not aligned at cache lines (postfix)
aarch64 RHEL7 and Centos7 internal compiler error here, so we carry
another pragma enchantment like buf0lru.cc and row0ins.cc.
2022-04-21 19:12:24 +10:00
Marko Mäkelä
aec856073d WolfSSL v5.2.0-stable 2022-04-21 12:02:36 +03:00
Marko Mäkelä
f84b5d782a Fix clang -Wunused-but-set-variable 2022-04-21 11:35:07 +03:00
Marko Mäkelä
394784095e Merge 10.3 into 10.4 2022-04-21 11:33:59 +03:00
Marko Mäkelä
4730314a70 MDEV-28369 ibuf_bitmap_mutex is an unnecessary contention point
The only purpose of ibuf_bitmap_mutex is to prevent a deadlock between
two concurrent invocations of ibuf_update_free_bits_for_two_pages_low()
on the same pair of bitmap pages, but in opposite order.
The mutex is unnecessarily serializing the execution of the function
even when it is being invoked on totally different tablespaces.
To avoid deadlocks, it suffices to ensure that the two page latches
are being acquired in a deterministic (sorted) order.
2022-04-21 09:15:18 +03:00
Daniel Black
580cbd18b3 Merge branch 10.4 into 10.5
A few of constaint -> constraint
2022-04-21 15:47:03 +10:00
Daniel Black
b8398ee1fd MDEV-21208: mysql_tzinfo_to_sql does not work in strict mode
Remove Warning that occured by doing an ALTER TABLE ... ORDER
BY on an InnoDB table.

Reviewed by Brandon Nesterenko
2022-04-21 14:59:29 +10:00
Daniel Black
13e77930e6 MDEV-28263: mariadb-tzinfo-to-sql improve wsrep and binlog cases
The --skip-write-binlog message was confusing that it only had
an effect if the galera was enabled. There are uses beyond galera
so we apply SET SESSION SQL_LOG_BIN=0 as implied by the option
without being conditional on the wsrep status.

We also with --skip-write-binlog actually check the session @@WSREP_ON
variable rather than the global server variable.

Since 10.6, the wsrep_mode could replicate Aria and MyISAM, in which
case no change to innodb and back is needed.

By removing the conditions, we can use LOCK TABLES in a general case
improving the load speed of Aria (MDEV-23326), regardless of the
skip-write-binlog flag. The only case where we don't use LOCK TABLES is
when we are replicating via Innodb, because wsrep_on=1 and wsrep_mode
doesn't contain REPLICATE_ARIA{,MYISAM}. This uses an Innodb transaction
instead. When replicating via InnoDB we change the table engine type
back to what it was originally.

By removing the \d and other syntax that requires parsing by
the mariadb client, we can use the generated SQL more generally, like
in the embedded server.

We also save and restore the SQL_LOG_BIN and WSREP_ON session server
variables so this can be included in the same session as other data
without taking into changes in state.

Remove wsrep.mysql_tzinfo_to_sql_symlink{,_skip} tests as they offered
no additional coverage beyond main.mysql_tzinfo_to_sql_symlink (no
server testing was done).

Add galera.mariadb_tzinfo_to_sql to actually test the replication
of tzinfo data through galera.

The conditional executable comment around /*M!100602 ...
START TRANSACTION .. LOCK TABLES.. */ is so that we can provide tzinfo
files (MDEV-27113, MDBF-389) and in the case that a user uses it on a
pre-10.6 server version it will still work. Both START TRANSACTION and
LOCK TABLES are not supported in prepared statements in MariaDB versions
earlier than 10.6.2.

Reviewed by Brandon Nesterenko
2022-04-21 14:59:29 +10:00
Haidong Ji
d7189fbcb4 MDEV-28227 Error message Chinese translation
- Simplified Chinese translation added
- Character encoding is gdk
-- gdk covers more characters
-- gdk includes both Simplified and Traditional
-- best option I think, may need to work along with other locale
settings
- Other cleanup
-- Within each error, messages are sorted according to language code
-- More consistent formatting (8 spaces proceeding each translation)
-- jps removed as duplicate of jpn translation

This should be a good starting point. More refinement is appreciated,
and needed down the road.

English "containt" (sic) spelling fixes on ER_FK_NO_INDEX_{CHILD,PARENT}
resulting in mtr test case adjustments.

Edited/reviewed by Daniel Black
2022-04-21 13:57:08 +10:00
Thirunarayanan Balathandayuthapani
372b0e6355 MDEV-20194 Warnings inconsistently issued upon CHECK on table from older versions
The following condition has to added:
1) InnoDB fails to include the offset of the node pointer field
in non-leaf record for redundant row format.

2) If the Fixed length field does have only prefix length then
calculate the field maximum size as prefix length.

- Added the test case to test (2) and to check maximum number of
fields can exist in the index.
2022-04-20 19:55:17 +05:30
Thirunarayanan Balathandayuthapani
d91c268096 MDEV-28370 ASAN failure: InnoDB fails to free the memory allocated on dict_load_indexes()
- While loading the indexes, InnoDB fails to clear the index if the
system record has TEMP_INDEX_PREFIX_STR. This lead to ASAN failure.
The leak was introduced by commit cc2ddde4d8
(MDEV-18518)
2022-04-20 19:14:55 +05:30
Daniele Sciascia
11e5aba792 MDEV-26575 Crash on shutdown after starting an XA transaction
Disallow XA when Galera library is loaded.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2022-04-20 10:41:19 +03:00
Jan Lindström
84b135065c MDEV-28314 : The Galera cluster primary node goes into hang mode when innodb_encryption_threads is enabled
When we enable writes after Galera SST srv_n_fil_crypt_threads needs
to be set temporally to 0 (as was done when writes were disabled)
to make sure that encryption threads will be really started based
on old value of encryption threads.

Fix provided by Marko Mäkelä.
2022-04-20 07:42:21 +03:00
Marko Mäkelä
d1edb011ee Cleanup: Remove os0thread
Let us use the common pthread_t wrapper for Microsoft Windows.

This fixes up commit dbe941e06f
2022-04-19 13:49:52 +03:00
Marko Mäkelä
b3c3291f0b MDEV-24176 fixup: GCC -Wmaybe-uninitialized 2022-04-19 10:29:21 +03:00
Jan Lindström
89b1172373 MDEV-28329 : galera_3nodes.galera_garbd_backup fails with extra connection lines
Add missing connection lines to result set
2022-04-19 08:44:51 +03:00
Aleksey Midenkov
08c7ab404f MDEV-24176 Server crashes after insert in the table with virtual
column generated using date_format() and if()

vcol_info->expr is allocated on expr_arena at parsing stage. Since
expr item is allocated on expr_arena all its containee items must be
allocated on expr_arena too. Otherwise fix_session_expr() will
encounter prematurely freed item.

When table is reopened from cache vcol_info contains stale
expression. We refresh expression via TABLE::vcol_fix_exprs() but
first we must prepare a proper context (Vcol_expr_context) which meets
some requirements:

1. As noted above expr update must be done on expr_arena as there may
be new items created. It was a bug in fix_session_expr_for_read() and
was just not reproduced because of no second refix. Now refix is done
for more cases so it does reproduce. Tests affected: vcol.binlog

2. Also name resolution context must be narrowed to the single table.
Tested by: vcol.update main.default vcol.vcol_syntax gcol.gcol_bugfixes

3. sql_mode must be clean and not fail expr update.

sql_mode such as MODE_NO_BACKSLASH_ESCAPES, MODE_NO_ZERO_IN_DATE, etc
must not affect vcol expression update. If the table was created
successfully any further evaluation must not fail. Tests affected:
main.func_like

Reviewed by: Sergei Golubchik <serg@mariadb.org>
2022-04-18 12:44:27 +03:00
Aleksey Midenkov
c02ebf3510 MDEV-24176 Preparations
1. moved fix_vcol_exprs() call to open_table()

mysql_alter_table() doesn't do lock_tables() so it cannot win from
fix_vcol_exprs() from there. Tests affected: main.default_session

2. Vanilla cleanups and comments.
2022-04-18 12:44:27 +03:00
Oleg Smirnov
39cc2545af MDEV-24529 Assertion failed in vers_select_conds_t::print
This commit adds processing of SYSTEM_TIME_BEFORE and SYSTEM_TIME_HISTORY
to vers_select_conds_t::print().
2022-04-18 11:19:34 +03:00
Oleg Smirnov
7498978e6a MDEV-27699 ANALYZE FORMAT=JSON fields are incorrect for UNION ALL queries
UNION ALL queries are a subject of optimization introduced in MDEV-334
when creation of a temporary table is skipped.
While there is a check for this optimization in Explain_union::print_explain()
there was no such in Explain_union::print_explain_json(). This resulted in
printing irrelevant data like:
  "union_result": {
    "table_name": "<union2,3>",
    "access_type": "ALL",
    "r_loops": 0,
    "r_rows": null
in case when creation of the temporary table was actually optimized out.
This commits adds a check whether the temporary table was actually created
during the UNION ALL processing and eliminates printing of the irrelevant data.
2022-04-18 07:50:14 +03:00
Daniele Sciascia
8a322b6b0c Disable galera_bf_abort_ps_bind under --ps-protocol 2022-04-16 08:20:54 +03:00
Marko Mäkelä
7da351d804 Merge 10.5 into 10.6 2022-04-15 21:02:10 +03:00
Marko Mäkelä
a9adfc0f68 MDEV-9948 Failing assertion: new_state->key_version != ENCRYPTION_KEY_VERSION_INVALID in fil0crypt.cc
The test encryption.innodb-redo-nokeys did not actually test
recovery without valid keys, because due to the setting
innodb_encrypt_tables, InnoDB refused to start up at all,
without even attempting any crash recovery.

fil_ibd_load(): If the encryption key is not available,
refuse to load the file.
2022-04-15 20:28:33 +03:00
Nayuta Yanagisawa
4abb023ba5 Spider: disable spider/bugfix.mdev_27239 2022-04-15 18:53:51 +09:00
Alexander Barkov
83516a33a8 An additional patch for MDEV-27690 Crash on CHARACTER SET csname COLLATE DEFAULT in column definition
Applying the fix to sql_yacc_ora.yy. Adding a test for sql_mode=ORACLE.
2022-04-14 12:22:28 +04:00
Alexander Barkov
9d734cdd61 Merge remote-tracking branch 'origin/10.2' into 10.3 2022-04-14 11:50:34 +04:00
Marko Mäkelä
2aed566d22 Cleanup: alignas(CPU_LEVEL1_DCACHE_LINESIZE)
Let us replace all use of MY_ALIGNED in InnoDB with C++11 alignas.

CACHE_LINE_SIZE: Replaced with CPU_LEVEL1_DCACHE_LINESIZE.
2022-04-14 10:40:26 +03:00
Marko Mäkelä
03f9bb8c90 MDEV-28313: Shrink ReadView::m_mutex
A few PERFORMANCE_SCHEMA instrumentation keys were not exposed
in all_innodb_mutexes[]. Let us remove them.

The keys fts_pll_tokenize_mutex_key and read_view_mutex_key were
internally used. Let us make ReadView::m_mutex use the simpler
and smaller srw_mutex, hoping to improve memory access patterns.
2022-04-14 10:21:43 +03:00
Marko Mäkelä
8074ab5784 MDEV-28313: Shrink rw_trx_hash_element_t::mutex
The element mutex is unnecessarily large. The PERFORMANCE_SCHEMA
instrumentation was not even enabled.
2022-04-14 10:10:44 +03:00
Marko Mäkelä
0cd2e6c614 MDEV-28313: InnoDB transactions are not aligned at cache lines
trx_lock_t: Remove byte pad[256] and use
alignas(CPU_LEVEL1_DCACHE_LINESIZE) instead.

trx_t: Declare n_ref (the first member) aligned at cache line.

Pool: Assert that the sizes are multiples of
CPU_LEVEL1_DCACHE_LINESIZE, and invoke an aligned allocator.
2022-04-14 10:06:34 +03:00
Sergei Golubchik
f7f0bc748e cleanup: un-inline dtype_get_mblen()
per Marko request
2022-04-13 17:50:00 +02:00
Sergei Golubchik
fc8396448c MDEV-27767 poor scaling with InnoDB and utf8mb3 because of charset stats
Access the all_charsets[] array directly in a hot loop.
This avoids get_charset() that increments a shared counter via
my_collation_statistics_inc_use_count(), causing a scalability issue.

Instead, call get_charset() when a table is opened (and InnoDB
fills in dtype_t values) - this is enough, as charset is marked
ready (MY_CS_READY) only once, on the first get_charset() call,
after that it can be accessed directly.

This also fixes a potential bug in InnoDB. It used to access charsets
directly when filling dtype_t values. This wasn't preceded by any
get_charset() call inside InnoDB. Normally the server would call
get_charset() on reading the frm, but if the table was first opened
from a purge thread InnoDB could, theoretically, access a not-ready
charset in innobase_get_cset_width().
2022-04-13 17:50:00 +02:00
Oleksandr Byelkin
f130a5ea3c MDEV-28266 Crash in Field_string::type_handler when calling procedures
on_table_fill_finished() should always be done at the end of open()
even if result is not Select_materialize but (for example)
Select_fetch_into_spvars.
2022-04-13 15:37:46 +02:00
Oleksandr Byelkin
5615a78a69 MDEV-28266 Crash in Field_string::type_handler when calling procedures
on_table_fill_finished() should always be done at the end of open()
even if result is not Select_materialize but (for example)
Select_fetch_into_spvars.
2022-04-13 15:11:59 +02:00
Marko Mäkelä
e4835bf572 MDEV-28304 innodb.instant_alter,8k.rdiff does not apply on FreeBSD
This is conceptually a backport of
commit 6f4740fde7
(re-recording the .rdiff files).
2022-04-13 09:23:52 +03:00
Julius Goryavsky
42908dc5fb MDEV-26171: wsrep_sst_receive_address does not parse IPv6 address correctly
This commit fixes problems with parsing ipv6 addresses given via
the wsrep_sst_receive_address and wsrep_node_address options.

Also, this commit removes extra lines in the configuration files
in the mtr test suites for Galera related to these parameters.
2022-04-12 17:14:39 +02:00
Marko Mäkelä
cdc0bbdd87 Merge 10.5 into 10.6 2022-04-12 14:53:58 +03:00
Sergei Golubchik
d31732e02e MDEV-27834 Incorrect provides of MariaDB-shared for CentOS8
fix bad merge
2022-04-12 13:38:41 +02:00
Sergei Golubchik
bbdec04d59 MDEV-24317 Data race in LOGGER::init_error_log at sql/log.cc:1443 and in LOGGER::error_log_print at sql/log.cc:1181
don't initialize error_log_handler_list in set_handlers()
* error_log_handler_list is initialized to LOG_FILE early, in init_base()
* set_handlers always reinitializes it to LOG_FILE, so it's pointless
* after init_base() concurrent threads start using sql_log_warning,
  so following set_handlers() shouldn't modify error_log_handler_list
  without some protection
2022-04-12 13:07:20 +02:00
Alexander Barkov
e41500e4f2 MDEV-26128 type_set and type_enum are broken
The problem was not with the server behavior,
it was with wrong test results recorded.

Enabling both type_set and type_enum.

type_set.result was OK. The test did not fail.

type_enum.result was incorrectly recorded in this commit:
  eb483c5181

Recording correct results.
2022-04-12 14:26:02 +04:00
Marko Mäkelä
ca3bbf4c0c Merge 10.5 into 10.6 2022-04-12 09:26:02 +03:00
Tuukka Pasanen
bc75f7ed6d MDEV-28194: Remove unneeded path from MariaDB server postinst script
Remove unndeeded paths from mariadb-server-10.6.postinst as
it's already in export PATH and it fires 'command-with-path-in-maintainer-script'
in lintian checks
2022-04-12 13:28:30 +10:00
Shunpoco
767d8d8335 MDEV-27448 MTR returns success (zero) upon invalid option
I change from `exit;` to `exit(1);` on a function `usage()`.
When we try to run mtr with a wrong option, a function `usage()` is called with the wrong option as its argument. In this case, because the function call `exit` in a first if statement, we get exit status 0.
2022-04-12 08:17:32 +10:00
Rucha Deodhar
3327bb6098 MDEV-22266: Diagnostics_area::sql_errno() const: Assertion
`m_status == DA_ERROR' failed on SELECT after setting tmp_disk_table_size.

Analysis: Mismatch in number of warnings between "194 warnings" vs
"64 rows in set" is because of max_error_count variable which has default
value of 64.
About the corrupted tables, the error that occurs because of insufficient
tmp_disk_table_size variable is not reported correctly and we continue to
execute the statement. But because the previous error (about table being
full)is not reported correctly, this error moves up the stack and is
wrongly reported as parsing error later on while parsing frm file of one
of the information schema table. This parsing error gives corrupted table
error.
As for the innodb error, it occurs even when tmp_disk_table_size is not
insufficient is default but the internal error handler takes care of it
and the error doesn't show. But when tmp_disk_table_size is insufficient,
the fatal error which wasn't reported correctly moves up the stack so
internal error handler is not called. So it shows errors.
Fix: Report the error correctly.
2022-04-12 01:22:51 +05:30
Alexander Barkov
833f4486cf MDEV-27690 Crash on CHARACTER SET csname COLLATE DEFAULT in column definition
This is a temporary fix for 10.2.
This problem was permanently fixed in 10.9 under terms of MDEV-27743.

This patch should propagate up to 10.8 then null-merged to 10.9.
2022-04-11 19:45:31 +04:00