Commit graph

194251 commits

Author SHA1 Message Date
Brandon Nesterenko
555c12a541 MDEV-21087/MDEV-21433: ER_SLAVE_INCIDENT arrives at slave without failure specifics
Problem:
=======

This patch addresses two issues:

 1. An incident event can be incorrectly reported for transactions
which are rolled back successfully. That is, an incident event
should only be generated for failed “non-transactional transactions”
(i.e., those which modify non-transactional tables) because they
cannot be rolled back.

 2. When the mariadb slave (error) stops at receiving the incident
event there's no description of what led to it. Neither in the event
nor in the master's error log.

Solution:
========

Before reporting an incident event for a transaction, first validate
that it is “non-transactional” (i.e. cannot be safely rolled back).
To determine if a transaction is non-transactional,
  lex->stmt_accessed_table(LEX::STMT_WRITES_NON_TRANS_TABLE)
is used because it is set previously in
THD::decide_logging_format().

Additionally, when an incident event is written, write an error
message to the server’s error log to indicate the underlying issue.

Reviewed by:
===========
Andrei Elkin <andrei.elkin@mariadb.com>
2022-07-25 16:26:53 -06:00
Rucha Deodhar
46ff660883 This commit is a fixup for MDEV-28762 2022-07-25 22:13:40 +05:30
Marko Mäkelä
f1c8749f46 Fix DBUG_ENTER/return mismatch
Spotted by Thirunarayanan Balathandayuthapani.
2022-07-25 15:58:16 +03:00
Marko Mäkelä
8aa37c264f MDEV-28980 InnoDB: Failing assertion: len <= MAX_TABLE_NAME_LEN
dict_load_foreigns(): Use a correctly sized buffer for the maximum-length
SYS_FOREIGN.ID. In case of overflow, do not crash the server but instead
return DB_CORRUPTION.
2022-07-25 10:28:45 +03:00
Brad Smith
e55c3dc33f Fix building Connect storage engine ioapi code on OpenBSD / NetBSD / DragonFly 2022-07-25 15:06:32 +10:00
Rucha Deodhar
95989e8211 MDEV-28762: recursive call of some json functions without stack control
This commit is a fixup for MDEV-28762

    Analysis: Some recursive json functions dont check for stack control
    Fix: Add check_stack_overrun(). The last argument is NULL because it is not
    used
2022-07-23 23:02:12 +05:30
Oleksandr Byelkin
4bc34ef36f Merge branch 'merge-zlib' into 10.3 2022-07-22 21:08:55 +02:00
haomi123
7b0e68b8a2 fix DBUG_ENTER awake_no_mutex 2022-07-22 14:59:18 +02:00
Oleksandr Byelkin
8b6739e4e3 1.2.12 2022-07-22 08:32:45 +02:00
Oleksandr Byelkin
3fb50da076 1.2.11 2022-07-21 15:27:25 +02:00
Rucha Deodhar
dbe39f14fe MDEV-28762: recursive call of some json functions without stack control
Analysis: Some recursive json functions dont check for stack control
Fix: Add check_stack_overrun(). The last argument is NULL because it is not
used
2022-07-20 19:24:48 +05:30
Aleksey Midenkov
1848804840 MDEV-29023 MTR hangs after multiple failures
Passing $opt_parallel as $childs is wrong: child can be killed before
it connects and you will never decrement $childs for this.

Another problem is (and that is the cause of this bug): child can be
killed and never close server socket. This can happen f.ex. after
unmaskable KILL signal. In such case the socket is closed by reaping
the child but that never happens inside reading the socket loop in
run_test_server().

The proper design is the waitless reap of children inside the socket
loop and if there is no more children we finish the socket loop. Since
there is Windows variation where we don't control the children via
waitpid(), all the clients must normally close the socket and only
this can finish the socket loop. For Unix variation we reckon that
case as all children closed the socket but not all yet died and for
that we do final waiting waitpid() (was done before the patch as
well).

To be more complete, we now handle 3 end-of-game scenarios in Unix:

   1. all children closed socket, all children died: everything is
      handled by the socket loop;

   2. all children closed socket, not all yet died: we wait for alive
      children to die after exiting the socket loop;

   3. not all children closed socket, all children died: everything is
      handled by the socket loop.

For Windows end-of-game scenario is only one:

   All children close the socket.
2022-07-18 23:16:18 +03:00
Aleksey Midenkov
7ca5c7d8f9 MDEV-29023 waitpid() cleanup
The case for "Unknown process $ret_pid exited" is never known to be
valid. Such state is not documented for waitpid().
2022-07-18 23:16:18 +03:00
Aleksey Midenkov
1d307ed48a MDEV-29025 MTR doesn't print stack trace for bootstrap crashes 2022-07-18 23:16:18 +03:00
Aleksey Midenkov
1bdcffb028 MDEV-29025 Refactoring: moved out core_wanted() out of mysql-test-run.pl 2022-07-18 23:16:17 +03:00
Aleksey Midenkov
e9be5428a2 MDEV-28931 MTR prints detailed stack trace unconditionally
66832e3a introduced change that prints core dumps in very detailed
format. That's completely out of user-friendliness but serves as a
measure for debugging hard-reproducible bugs.

The proper way to implement this:

  1. it must be controlled by command-line and environment variable;
  2. detailed traces must be default for buildbots only, for user
     invocations normal stack traces should be printed.

Options for control are: MTR_PRINT_CORE and --print-core that accept
the following values:

  no	         Don't print core
  short	       	 Print stack trace of failed thread
  medium	 Print stack traces of all threads
  detailed       Print all stack traces with debug context
  custom:<code>  Use debugger commands <code> to print stack trace

Default setting is: short (see env_or_default() call in pre_setup())

For environment variable wrong values are silently ignored (falls back
to default setting, see env_or_default()).

Command-line option --print-core (or -C) overrides environment
variable. Its default value is 'short' if not specified explicitly
(same env_or_default() call in pre_setup()). Explicit values are
checked for validity.

--print-method option can specify by which debugger we print
cores. For Windows there is only one choice: cdb. For Unix the values
are: gdb, dbx, lldb, auto. Default value is: auto

In 'auto' we try to use all possible debuggers until success.
2022-07-18 23:16:17 +03:00
Aleksey Midenkov
220fb6797b MDEV-28931 Debugger.pm readability fix
setup_boot_args(), setup_client_args(), setup_args() traversing
datastructures on each invocation. Even if performance is not
important to perl script (though it definitely saves some CO2), this
nonetheless provokes some code-reading questions. Reading and
debugging such code is not convenient.

The better way is to prepare all the data in advance in an easily
readable form as well as do the validation step before any further
processing.

Use mtr_report() instead of die() like the other code does.

TODO: do_args() does even more data processing magic. Prepare that
data according the above strategy in advance in pre_setup() if possible.
2022-07-18 23:16:17 +03:00
Aleksey Midenkov
ce7820eb83 MDEV-28931 --verbose option is too verbose
GetOpt::Long bundling option for convenient one-char verbosity levels:

  -v    General verbosity (file and execute operations)
  -vv   High verbosity (algorithmic considerations)
  -vvv  Debug verbosity (anything else)
2022-07-18 23:16:17 +03:00
Aleksey Midenkov
83f7d25c44 MDEV-28931 Cleanup: try GDB to print core first
Do we still need this Sun Studio hack?
2022-07-18 23:16:17 +03:00
Vladislav Vaintroub
990ddaba1e Windows - reduce irrelevant CMake system checks 2022-07-18 14:59:07 +02:00
Vladislav Vaintroub
b3f0acf510 MDEV-27686 Moving libexecinfo out of FreeBSD / OpenBSD specific CMake code
On all Unix platforms, link libexecinfo as system library,
if it contains backtrace_symbols_fd function, and libc does not contain
this function

Also remove cmake/os/OpenBSD.cmake, as after the fix it serves no purpose.
2022-07-17 15:14:52 +02:00
Alexey Botchkov
8911823f65 MDEV-26546 SIGSEGV's in spider_db_connect on SHOW TABLE and spider_db… …_mbase::connect (and SIGSEGV's in check_vcol_forward_refs and inline_mysql_mutex_lock)
Not the SPIDER issue - happens to INSERT DELAYED.
the field::make_new_field does't copy the LONG_UNIQUE_HASH_FIELD
flag to the new field. Though the Delayed_insert::get_local_table
copies the field->vcol_info for this field. Ad a result
the parse_vcol_defs doesn't create the expression for that column
so the field->vcol_info->expr is NULL. Which leads to crash.
Backported fix for this from 10.5 - the flagg added in the
Delayed_insert::get_local_table.

Another problem with the USING HASH key is thst the
parse_vcol_defs modifies the table->keys content. Then the same
parse_vcol_defs is called on the table copy that has keys already
modified. Backported fix for that from 10.5 - key copying added
tot the Delayed_insert::get_local_table.

Finally - the created copy has to clear the expr_arena as
this table is not in the thd->open_tables list so won't be
cleared automatically.
2022-07-17 01:10:43 +04:00
Oleksandr Byelkin
92a3280998 table_count was present twice in one class of LEX.
Remove table_count from Query_tables_list (not used, moved to MYSQL_LOCK).

Rename table_count from LEX to avoid mixing it with other counters of tables.
2022-07-14 09:46:06 +02:00
Oleg Smirnov
49e14000ee MDEV-26427 MariaDB Server SEGV on INSERT .. SELECT
1. For INSERT..SELECT statements: don't include table/view the data
   is inserted into in the list of leaf tables
2. Remove duplicated and dead code related to table_count
2022-07-14 11:07:24 +07:00
Igor Babaev
65cc89ed9e MDEV-29088 Server crash upon CREATE VIEW with unknown column in ON condition
This bug caused crashes when the server executed such a CREATE VIEW
statement whose view specification contained a reference to an unknown
column in a subquery used in ON condition.
The cause of this bug is quite similar to the cause of the bug MDEV-26412.
The fix of this bug is quite similar to the fix for MDEV-26412.

Approved by Sergey Petrunia <sergey@mariadb.com>
2022-07-13 11:14:45 -07:00
Brandon Nesterenko
02e85aeafd MDEV-28487: sequences not respect value of binlog_row_image with select nextval(seq_gen)
Problem:
========
When using sequences, the function
sequence_definition::write(TABLE *table, bool all_fields)
is used to save DML/DDL updates to sequence tables (e.g. nextval,
setval, and alter). Prior to this patch, the value all_fields was
always false when invoked via nextval and setval, which forced the
bitmap to only include changed columns.

Solution:
========
Change all_fields when invoked via nextval and setval to be reliant
on binlog_row_image, such that it is false when binlog_row_image is
MINIMAL, and true otherwise.

Reviewed By:
===========
Andrei Elkin <andrei.elkin@mariadb.com>
2022-07-13 09:03:32 -06:00
Dmitry Shulga
f439cfdf93 MDEV-22001: Server crashes in st_select_lex_unit::exclude_level upon execution of SP
Running some statements that use IN subqueries outside context of a regular
query could result in server abnormal termination.

The reason for failure is that internal structures SELECT_LEX/SELECT_LEX_UNIT
created on behalf of parsed query were initialized incorrectly. Incorrect
initialization of the structures SELECT_LEX/SELECT_LEX_UNIT was introduced
by the commit de745ecf29
(MDEV-11953: support of brackets in UNION/EXCEPT/INTERSECT operations)
pushed into 10.4, that is the reason this bug report is not reproduced in 10.3.

To fix the issue the method SLECTE_LEX::register_unit is used for proper
initialization of the data structures SELECT_LEX/SELECT_LEX_UNIT. Additionally,
the method SELECT_LEX::get_slave() was removed from the source code base
since for those use cases where it is used it can be replaced by the method
first_inner_unit().
2022-07-12 17:18:48 +07:00
Oleksandr Byelkin
8494758e8e MDEV-26433 assertion: table->get_ref_count() == 0 in dict0dict.cc line 1915
Close handlers in THD::drop_temporary_table.
2022-07-12 11:12:55 +02:00
Sergei Golubchik
96be3fe841 MDEV-21445 update test results
followup for 57f5c319af
2022-07-11 23:50:56 +02:00
Marko Mäkelä
b817afaa1c MDEV-28689, MDEV-28690: Remove ctrl_mutex
This reverts the revert 4f62dfe676
and fixes the hang that was introduced when ctrl_mutex was removed.

The test mariabackup.compress_qpress covers this code, but the
test is skipped if a stand-alone qpress executable is not available.
It is not available in many software repositories, possibly because
the code base has not been updated since 2010.

This was tested with an executable that was compile from the source
code at http://www.quicklz.com/qpress-11-source.zip (after adding
a missing #include <unistd.h> for the definition of isatty()).

Compared to the grandparent commit (before the revert), the changes
are as follows:

comp_thread_ctxt_t::done_cond: A separate condition for completed
compression, signaling that thd->to_len has been updated.

compress_write(): Replace some threads[i] with thd.
Reset thd->to_len = 0 after consuming the compressed data.

compress_worker_thread_func(): After consuming the uncompressed
data, set thd->data_avail = FALSE. After compressing, signal
thd->done_cond.
2022-07-11 21:00:18 +03:00
Vladislav Vaintroub
4f62dfe676 Revert "MDEV-28689, MDEV-28690: Incorrect error handling for ctrl_mutex"
This reverts commit 863c3eda87.
2022-07-11 15:00:34 +02:00
Sergei Golubchik
7598ef4b26 MDEV-28197 Linux mariadb-client build does not accept Unicode characters
Apparently newer libedit is readline-compatible enough
to be detected as a readline, with USE_NEW_READLINE_INTERFACE defined
and USE_LIBEDIT_INTERFACE not defined.

Let's set the locale unconditionally, independently from the
readline/libedit variant. It's already happening anyway now,
unless one specifies --default-character-set explicitly.
2022-07-10 22:37:13 +02:00
Daniel Black
fc456bc97e MDEV-27593 InnoDB handle AIO errors - more detailed assertion
Step 1 in handling InnoDB AIO assertions better is to get
more detail of the cases of error.

This doesn't resolve MDEV-27593, but increases the level
of information in the assertion.
2022-07-09 12:34:02 +10:00
Vladislav Vaintroub
781948a19b MDEV-22954 fix sporadic failure of main.mdl
Fix the side effect of MDEV-4750 (reenabling innodb_stats_persistent),
so that sporadic MDL acquisition for this table does not interfere with
SELECT from information_schema.metadata_lock_info
2022-07-08 18:37:12 +02:00
Sergei Petrunia
0e9a255ec8 MDEV-28871: Assert ... failed in JOIN::dbug_verify_sj_inner_tables...
optimize_semi_joins() calls update_sj_state() to update semi-join
optimization state in the JOIN class.

greedy_search() algorithm considers different join prefixes,
and then picks one table to put into the join prefix.
Most of the semi-join optimization state is in the table's entry
in the join->positions[cur_prefix_size].

However, it also needs to call update_sj_state() to update the
semi-join optimization state in the JOIN class.

There is one exception, which is the cause of this bug: when we're
inside optimize_semi_join_nests() and are optimizing a subquery,
optimize_semi_joins() does nothing, it doesn't call update_sj_state().

greedy_search() must not do that either.
2022-07-07 22:15:42 +03:00
Vladislav Vaintroub
66c06735a2 MDEV-28746 Wrong error code ER_BAD_DB_ERROR for long filenames
Add check for path length if CreateFile fails with ERROR_PATH_NOT_FOUND.
2022-07-07 15:14:14 +02:00
Alexander Barkov
d6e80c21d6 MDEV-25492 BETWEEN clause returns incorrect results on quoted 64-bit ints
This problem was earlier fixed by the patch for MDEV-21445.
2022-07-06 16:36:36 +04:00
Alexander Barkov
57f5c319af MDEV-21445 Strange/inconsistent behavior of IN condition when mixing numbers and strings 2022-07-06 15:42:21 +04:00
Alexander Barkov
bdc1134dea MDEV-29041 Redundant truncation warning on CAST(string_column AS DECIMAL) 2022-07-06 15:05:25 +04:00
Sergei Golubchik
6313702278 MDEV-26568 RPM logic prohibiting server major upgrade no longer works as expected
prevent %{VERSION} in the shell command to be expanded by rpmbuild
2022-07-05 21:09:59 +02:00
Sergei Golubchik
0fca5068a8 MDEV-28234 Change maturity of plugins for July 2022 Releases
mysql_json: GAMMA -> STABLE
2022-07-05 19:07:36 +02:00
Nayuta Yanagisawa
2f3f1cd05b MDEV-26544 Assertion `part_share->auto_inc_initialized' failed in ha_partition::get_auto_increment on INSERT
The partition storage engine ignores return (error) values of
handler::info(). As a result, a query that should be aborted is
not aborted and then the server violates the assertion.
2022-07-05 23:07:49 +09:00
Nayuta Yanagisawa
4bc7c03b5f MDEV-29011 Server crash in spider_db_open_item_cond with XOR operator
Item_func_xor and Item_cond are both derived class of Item_bool_func.
Spider converts *Item_func_xor to *Item_cond and then calls the member
function argument_list(), which Item_func_xor does not implement.
2022-07-05 23:02:19 +09:00
Vladislav Vaintroub
8c8bd4bebf Fix typo in appveyor.yml
The convention used for options in mtr are 2 underscores
Prior to this patch, the "parallel" option used only one underscore
2022-07-04 19:24:58 +02:00
Vladislav Vaintroub
9d5718c9b9 MDEV-28648 main.ssl_timeout fails with OpenSSL 3.0.3
Depending on OpenSSL version, and at least in 3.0.3, the client-side socket
timeout is reported as generic error (SSL_ERROR_SYSCALL), losing further
details (both errno and GetLastError() return 0). This results in client
reporting "Unknown OpenSSL error" 2026, instead of another generic
"Lost connection to server during query" 2013

Adjusted test case.

Part of MDEV-29000
2022-07-04 12:49:11 +02:00
Honza Horak
ef65566981 MDEV-27778 md5 in FIPS crashes with OpenSSL 3.0.0
OpenSSL 3.0.0+ does not support EVP_MD_CTX_FLAG_NON_FIPS_ALLOW any longer.
In OpenSSL 1.1.1 the non FIPS allowed flag is context specific, while
in 3.0.0+ it is a different EVP_MD provider.

Fixes 

part of MDEV-29000
2022-07-04 12:49:11 +02:00
Oleksandr Byelkin
1dc09ce0fd Revert "don't build with OpenSSL 3.0, it doesn't work before MDEV-25785"
This reverts commit c9beef4315, because
we have OpenSSL 3.0 support here.

part of MDEV-29000
2022-07-04 12:49:11 +02:00
Vladislav Vaintroub
8a9c1e9ccf MDEV-25785 Add support for OpenSSL 3.0
Summary of changes

- MD_CTX_SIZE is increased

- EVP_CIPHER_CTX_buf_noconst(ctx) does not work anymore, points
  to nobody knows where. The assumption made previously was that
  (since the function does not seem to be documented)
  was that it points to the last partial source block.
  Add own partial block buffer for NOPAD encryption instead

- SECLEVEL in CipherString in openssl.cnf
  had been downgraded to 0, from 1, to make TLSv1.0 and TLSv1.1 possible
   (according to https://github.com/openssl/openssl/blob/openssl-3.0.0/NEWS.md
   even though the manual for SSL_CTX_get_security_level claims that it
   should not be necessary)

- Workaround Ssl_cipher_list issue, it now returns TLSv1.3 ciphers,
  in addition to what was set in --ssl-cipher

- ctx_buf buffer now must be aligned to 16 bytes with openssl(
  previously with WolfSSL only), ot crashes will happen

- updated aes-t , to be better debuggable
  using function, rather than a huge multiline macro
  added test that does "nopad" encryption piece-wise, to test
  replacement of EVP_CIPHER_CTX_buf_noconst

part of MDEV-29000
2022-07-04 12:49:11 +02:00
Marko Mäkelä
33f0270ebb Merge 10.4 into 10.5 2022-07-04 10:19:28 +03:00
Marko Mäkelä
9a0cbd31ce MDEV-26294 Duplicate entries in unique index not detected when changing collation
ha_innobase::check_if_supported_inplace_alter(): Refuse to change the
collation of a column that would become or remain indexed as part of
the ALTER TABLE operation.

In MariaDB Server 10.6, we will allow this type of operation;
that fix depends on MDEV-15250.
2022-07-04 08:04:44 +03:00