Commit graph

189890 commits

Author SHA1 Message Date
sjaakola
9377e9ba0c MDEV-21153 Replica nodes crash due to indexed virtual columns and FK cascading delete
Fix for MDEV-23033 fixes a problem in replication applying of transactions, which contain cascading foreign key delete for a table, which has indexed virtual column.
This fix adds slave_fk_event_map flag for table, to mark when the prelocking is needed for applying of a transaction.
See commit 608b0ee52e for more details.
However, this fix is targeted for async replication only, Rows_log_event::do_apply_event() has condition to rule out galera replication from the fix domain, and use cases suffering from MDEV-23033 and related MDEV-21153 will fail in galera cluster.

The fix in this commit removes the condition to rule out the setting of slave_fk_event_map flag from galera replication, and makes the fix in MDEV-23033 effective for galera replication as well.

However, the above fix has caused regressions for some galera_sr suite tests, which run tests for streaming replication.
This regression can be observed e.g. by: /mtr galera_sr.galera_sr_multirow_rollback  --mysqld=--slave_run_triggers_for_rbr=yes
These galera_sr suite tests were failing in last phase of replication applying, where actual transaction is already applied, and streaming replication related meta data needs to be updated in wsrep system tables.
Opening the wsrep system tables failed for corrupt data in THD::lex:query_tables_list. The fix in this commit uses back query table list for the duration of fragment update operation.

Finally, a mtr test for virtual column support has been added. galera.galera_virtual_column.test has as first test a scenario from MDEV-21153

new fix

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-01-20 08:11:13 +02:00
Igor Babaev
b22285e482 MDEV-16940 Server crashes in unsafe_key_update upon attempt to update view
through 2nd execution of SP

This bug caused a server crash on the second call of any stored procedure
that contained an UPDATE statement over a multi-table view reporting an
error message at the prepare stage.
On the first call of the stored procedure after reporting an error at
the preparation stage of the UPDATE statement finished without calling
the function SELECT_LEX::save_prep_leaf_tables() for the SELECT used as
the definition of the view. This left the SELECT_LEX structure used by
the UPDATE statement in an inconsistent state for second call of the stored
procedure.

Approved by Oleksandr Byelkin <sanja@mariadb.com>
2021-01-19 08:04:02 -08:00
Marko Mäkelä
3caccc7bcd Update InnoDB version number to 5.7.33
There are only two InnoDB changes between mysql-5.7.32 and mysql-5.7.33:

mysql/mysql-server@95dc4f5f08
duplicates commit 8e8e65ed1c (MDEV-10829).

mysql/mysql-server@26e849762f
could be an attempt to fix something that was fixed in
commit dc58987eb7 (MDEV-22765).
2021-01-19 14:43:04 +02:00
Marko Mäkelä
48ac7e1a42 MDEV-24609: innodb_io_capacity can exceed innodb_io_capacity_max
innodb_io_capacity_update(): When the requested innodb_io_capacity
exceeds innodb_io_capacity_max and is more than half the maximum,
do not double it for computing innodb_io_capacity_max.

This integer arithmetics overflow was introduced in
commit 0f32299437 (MDEV-7035).

No test case is added, because sizeof(ulong) varies between platforms.
2021-01-19 14:43:04 +02:00
Alexey Botchkov
959dfac4d0 MDEV-19723 Assertion je->state == JST_KEY' failed while SELECT ST_GEOMFROMGEOJSON() and Assertion !mysql_bin_log.is_open() || thd.is_current_stmt_binlog_format_row()'
The invalid GeoJSON case wasn't handled here.
2021-01-19 15:29:03 +04:00
Aleksey Midenkov
775aa6f08a MDEV-24403 Segfault on CREATE TABLE with explicit FTS_DOC_ID_INDEX by multiple fields
Ignore table->fts freed previously by create_table_info_t::create_table().
2021-01-19 14:25:51 +03:00
Thirunarayanan Balathandayuthapani
82d39d4374 MDEV-24491 db_name mismatch happens during virtual column computation
Database name mismatch happens while opening the table for
virtual column computation. Because table_name_parse() returns
the length of database and table name before converting the
filename to table name. This issue is caused by
commit 8b0d4cff07 (MDEV-15855).
Fix should be that table_name_parse() should return the length of
database and table name after converting the filename to table name.

Reviewed-by: Marko Mäkelä
2021-01-19 16:18:37 +05:30
Thirunarayanan Balathandayuthapani
479b4214fa MDEV-24547 Update fails when online alter does rollback due to MDL time out
When online alter rollbacks due to MDL time out, it doesn't mark the
index online status as ONLINE_INDEX_ABORTED. Concurrent update fails
to update the secondary index while building the entry.
InnoDB should check the online status of the secondary index before
building the secondary index entry.

Reviewed-by: Marko Mäkelä
2021-01-19 16:18:37 +05:30
Rucha Deodhar
94890a749a MDEV-24179: Assertion `m_status == DA_ERROR || m_status == DA_OK ||
m_status == DA_OK_BULK' failed in Diagnostics_area::message()

Analysis: Assertion failure happens because we reach the maximum limit to
examine rows.
Fix: Return the error state.
2021-01-19 16:16:03 +05:30
sjaakola
7d04ce6a2d MDEV-21153 Replica nodes crash due to indexed virtual columns and FK cascading delete
Fix for MDEV-23033 fixes a problem in replication applying of transactions, which contain cascading foreign key delete for a table, which has indexed virtual column.
This fix adds slave_fk_event_map flag for table, to mark when the prelocking is needed for applying of a transaction.
See commit 608b0ee52e for more details.
However, this fix is targeted for async replication only, Rows_log_event::do_apply_event() has condition to rule out galera replication from the fix domain, and use cases suffering from MDEV-23033 and related MDEV-21153 will fail in galera cluster.

The fix in this commit removes the condition to rule out the setting of slave_fk_event_map flag from galera replication, and makes the fix in MDEV-23033 effective for galera replication as well.

Finally, a mtr test for virtual column support has been added. galera.galera_virtual_column.test has as first test a scenario from MDEV-21153

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-01-19 12:12:10 +02:00
Dmitry Shulga
8bcddb02b7 MDEV-24577: Fix warnings generated during compilation of plugin/auth_pam/testing/pam_mariadb_mtr.c on FreeBSD
Compiler warnings generated on building MariaDB server for BSD has the same
reason as in case building is performed on MacOS. Both platforms do use
clang as a C/C++ compiler. So, fix the compiler warnings in case the compiler
is clang doesn't matter what kind of building platform do we use for building.

This is a follow-up patch for the following bug reports:
  MDEV-23564: CMAKE failing due to deprecated Apple GSS method
  MDEV-23935: Fix warnings generated during compilation of
              plugin/auth_pam/testing/pam_mariadb_mtr.c on MacOS
2021-01-19 16:08:46 +07:00
Marko Mäkelä
049811ec3c Merge 10.2 into 10.3 2021-01-19 08:31:03 +02:00
sjaakola
beaea31ab1 MDEV-23851 BF-BF Conflict issue because of UK GAP locks
Some DML operations on tables having unique secondary keys cause scanning
in the secondary index, for instance to find potential unique key violations
in the seconday index. This scanning may involve GAP locking in the index.
As this locking happens also when applying replication events in high priority
applier threads, there is a probabality for lock conflicts between two wsrep
high priority threads.

This PR avoids lock conflicts of high priority wsrep threads, which do
secondary index scanning e.g. for duplicate key detection.

The actual fix is the patch in sql_class.cc:thd_need_ordering_with(), where
we allow relaxed GAP locking protocol between wsrep high priority threads.
wsrep high priority threads (replication appliers, replayers and TOI processors)
are ordered by the replication provider, and they will not need serializability
support gained by secondary index GAP locks.

PR contains also a mtr test, which exercises a scenario where two replication
applier threads have a false positive conflict in GAP of unique secondary index.
The conflicting local committing transaction has to replay, and the test verifies
also that the replaying phase will not conflict with the latter repllication applier.
Commit also contains new test scenario for galera.galera_UK_conflict.test,
where replayer starts applying after a slave applier thread, with later seqno,
has advanced to commit phase. The applier and replayer have false positive GAP
lock conflict on secondary unique index, and replayer should ignore this.
This test scenario caused crash with earlier version in this PR, and to fix this,
the secondary index uniquenes checking has been relaxed even further.

Now innodb trx_t structure has new member: bool wsrep_UK_scan, which is set to
true, when high priority thread is performing unique secondary index scanning.
The member trx_t::wsrep_UK_scan is defined inside WITH_WSREP directive, to make
it possible to prepare a MariaDB build where this additional trx_t member is
not present and is not used in the code base. trx->wsrep_UK_scan is set to true
only for the duration of function call for: lock_rec_lock() trx->wsrep_UK_scan
is used only in lock_rec_has_to_wait() function to relax the need to wait if
wsrep_UK_scan is set and conflicting transaction is also high priority.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-01-18 08:09:06 +02:00
Jan Lindström
cf6114ebea MDEV-24432 : galera.galera_fk_cascade_delete_debug MTR failed: query 'reap' failed: 1205: Lock wait timeout exceeded
Add wait_conditions to verify correct database state before
next operation.
2021-01-15 07:00:09 +02:00
Jan Lindström
9e3aa83f01 MDEV-24443 : galera.lp1376747-4 MTR fails: Result length mismatch
Use debug_sync to force FTWRL to pause in correct state.
2021-01-15 07:00:09 +02:00
Alice Sherepa
db9b54f163 MDEV-12908 binlog_encryption.binlog_xa_recover, binlog.binlog_xa_recover failed in bb with extra checkpoint 2021-01-14 18:06:41 +01:00
Jan Lindström
b87828b6c8 MDEV-22285 : Assertion `xid_seqno > wsrep_seqno' failed in trx_rseg_update_wsrep_checkpoint on SET @@global.wsrep_start_position
Actual assertion mentioned on MDEV seems to be already fixed but
setting seqno to -2 will trigger a different assertion

mysqld: /home/jan/mysql/10.4-bugs/wsrep-lib/src/server_state.cpp:702: void wsrep::server_state::sst_received(wsrep::client_service&, int): Assertion `state_ == s_joiner || state_ == s_initialized' failed.

Fixed this by not allowing user to set seqno < -1 (-1 is special
seqno meaning undefined and seqno is initialized to it). MariaDB
releases 10.2 and 10.3 already do not allow to set seqno < -1.
2021-01-14 11:26:28 +02:00
Dmitry Shulga
f130adbf35 MDEV-23666: Assertion `m_cpp_buf <= ptr && ptr <= m_cpp_buf + m_buf_length' failed in Lex_input_stream::body_utf8_append
On parsing statements for which a starting backtick (`) delimiter doesn't have
a corresponding ending backtick, a current pointer to a position inside a
pre-processed buffer could go beyond the end of the buffer.

This bug report caused by the commit d496765903
  "MDEV-22022 Various mangled SQL statements will crash 10.3 to 10.5 debug builds".

In order to fix the issue both pointers m_ptr and m_cpp_ptr must be
rolled back to previous position in raw input and pre-processed input streams
correspondingly in case end of query reached during parsing.
2021-01-14 14:31:20 +07:00
Marko Mäkelä
c89f37983e MDEV-21478 fixup: Avoid a memory leak 2021-01-14 08:57:24 +02:00
Marko Mäkelä
ea9cd97f85 MDEV-24536 innodb_idle_flush_pct has no effect
The parameter innodb_idle_flush_pct that was introduced in
MariaDB Server 10.1.2 by MDEV-6932 has no effect ever since
the InnoDB changes from MySQL 5.7.9 were applied in
commit 2e814d4702.

Let us declare the parameter as deprecated and having no effect.
2021-01-13 18:55:56 +02:00
Marko Mäkelä
25db70f912 Fix innodb.innodb_mysql
It is unclear for how long the result has been broken, because
./mtr --big-test is not run regularly.
2021-01-13 18:54:53 +02:00
Rucha Deodhar
fb9a9599bc MDEV-24387: Wrong number of decimal digits in certain UNION/Subqery
constellation

Analysis: The decimals is set to NOT_FIXED_DEC for Field_str even if it is
NULL. Unsigned has decimals=0. So Type_std_attributes::decimals is set to 39
(maximum between 0 and 39). This results in incorrect number of decimals
when we have union of unsigned and NULL type.

Fix: Check if the field is created from NULL value. If yes, set decimals to 0
otherwise set it to NOT_FIXED_DEC.
2021-01-13 19:24:05 +05:30
Jan Lindström
4afab3c725 MDEV-18542 : galera_sr.galera-features#56: Test failure: signal 6; mysqltest: Can't connect to local MySQL server
Make test faster
2021-01-13 13:08:07 +02:00
Dmitry Shulga
9e4a5a81fc MDEV-24208 SHOW RELAYLOG EVENTS command is not supported in the prepared statement protocol yet
Added sending of metadata in response to preparing request for
the commands SQLCOM_SHOW_BINLOG_EVENTS, SQLCOM_SHOW_RELAYLOG_EVENTS
2021-01-13 16:16:13 +07:00
Jan Lindström
403818f466 MDEV-21523 : galera.MDEV-16509 MTR failed: timeout after 900 seconds: Can't connect to local MySQL server
Test uses Galera debug sync.
2021-01-13 08:45:03 +02:00
Olivier Bertrand
1d627ce47c Fix failed test 2021-01-13 00:23:08 +01:00
Olivier Bertrand
c9b5e5286b Fix failed test 2021-01-12 22:28:44 +01:00
Olivier Bertrand
347bce0201 - Remove static linkage to cpprestsdk when it is installed
modified:   storage/connect/CMakeLists.txt

- Continue BSON development
  modified:   storage/connect/bson.cpp
  modified:   storage/connect/bson.h
  modified:   storage/connect/bsonudf.cpp
  modified:   storage/connect/bsonudf.h
  added:      storage/connect/mysql-test/connect/r/bson_udf.result
  added:      storage/connect/mysql-test/connect/t/bson_udf.inc
  added:      storage/connect/mysql-test/connect/t/bson_udf.test
  added:      storage/connect/mysql-test/connect/t/bson_udf2.inc
2021-01-12 18:25:41 +01:00
Aleksey Midenkov
59998d3480 MDEV-23446 Missed error code fix 2021-01-12 18:44:24 +03:00
Varun Gupta
ab271ee7e2 MDEV-23826: ORDER BY in view definition leads to wrong result with GROUP BY on query using view
Introduced val_time_packed and val_datetime_packed  functions for Item_direct_ref
to make sure to get the value from the item it is referring to.

The issue for incorrect result was that the item was getting its value
from the temporary table rather than from the view.
2021-01-12 18:47:39 +05:30
Varun Gupta
3b94309a6c MDEV-23753: SIGSEGV in Column_stat::store_stat_fields
For EITS collection min and max fields are allocated for each column
that is set in the read_set bitmap of a table. This allocation of min and max
fields happens inside alloc_statistics_for_table.

For a partitioned table ha_rnd_init is called inside the function
collect_statistics_for_table which sets the read_set bitmap for the columns
inside the partition expression. This happens only when there is a write lock
on the partitioned table.
But the allocation happens before this, so min and max fields are not allocated
for the columns involved in the partition expression.
This resulted in a crash, as the EITS statistics were collected but there was
no min and max field to store the value to.

The fix would be to call ha_rnd_init inside the function alloc_statistics_for_table
that would make sure that min and max fields are allocated for the columns
involved in the partition expression.
2021-01-12 18:47:35 +05:30
Jan Lindström
7789726843 MDEV-24447 : galera.galera_toi_lock_shared MTR failed: WSREP: ALTER TABLE isolation failure
Add wait_condition so that INSERT is replicated before ALTER
and ALTER is replicated before we try to INSERT with new
number of columns.
2021-01-12 13:01:07 +02:00
Sergei Golubchik
3ffd5f28f0 MDEV-17227 Server crash in TABLE_SHARE::init_from_sql_statement_string upon table discovery with non-existent database
* failed init_from_binary_frm_image can clear share->db_plugin,
  don't use it on the error path
* cleanup the test a bit
2021-01-12 10:25:04 +01:00
Sergei Golubchik
0ee086838d MDEV-16735 mysql_upgrade failed
force alter_algorithm=DEFAULT in mysql_system_tables_fix.sql,
in case my.cnf sets it to something incompatible
2021-01-12 10:25:04 +01:00
Sergei Golubchik
f144ce2cfa MDEV-20763 Table corruption or Assertion `btr_validate_index(index, 0, false)' failed in row_upd_sec_index_entry with virtual column and EMPTY_STRING_IS_NULL SQL mode
unset empty_string_is_null mode when parsing generated columns in a table,
this mode affects pasring.
2021-01-12 10:25:04 +01:00
Sergei Golubchik
83bbe36831 fix sporadic failures of main.processlist_notembedded
the test was doing

--replace_result $con_id con_id
eval SHOW EXPLAIN FOR $con_id;

with the intention of replacing the variable part of the statement
in the result log. But actually replace_result replaces everything
that matches. In particular, when $con_id is 100, the warning

Note   1003    select sleep(100000)

becomes

Note   con_id3    select sleep(con_id000)
2021-01-12 10:25:04 +01:00
Sergei Golubchik
d463677f7e failing to parse an SP should not abort information_schema.routines 2021-01-12 10:25:03 +01:00
Sergei Golubchik
f7ff8f5dd9 MDEV-24524 Assertion `ls->length < 0xFFFFFFFFL && ((ls->length == 0 && !ls->str) || ls->length == strlen(ls->str))' failed in String::append on SELECT from I_S
don't expect return type of a stored function to be valid.
it's read from a table, so can be messed with.
it even can contain \0 bytes in the middle of the type name
2021-01-12 10:25:03 +01:00
Kentoku SHIBA
de5e5ab210 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
Add test cases.
2021-01-12 10:25:03 +01:00
Kentoku SHIBA
69c86abb64 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2021-01-12 10:25:03 +01:00
Varun Gupta
ffc384e044 MDEV-23804: Server crashes in st_select_lex::collect_grouping_fields_for_derived
The issue here was we were trying to push an extracted condition for a view into the
underlying table value constructor inside the view.
The fix would be to not push conditions into table value constructors.
2021-01-12 11:50:31 +05:30
Varun Gupta
1be707286e Added the test case for MDEV-23804 2021-01-12 11:38:26 +05:30
Sergei Golubchik
a216672dab MDEV-16341 Wrong length for USER columns in performance_schema tables
use USERNAME_CHAR_LENGTH and HOSTNAME_LENGTH for perfschema
USER and HOST columns
2021-01-11 21:54:48 +01:00
Sergei Golubchik
ad9a140d9b MDEV-14884 Failed to enable encryption of temporary files in mariadb 10.3.3
enable the encryption (and abort on failure) after printing --help,
not before
2021-01-11 21:54:48 +01:00
Sergei Golubchik
0d8bd7cc3a MDEV-18428 Memory: If transactional=0 is specified in CREATE TABLE, it is not possible to ALTER TABLE
* be strict in CREATE TABLE, just like in ALTER TABLE, because
  CREATE TABLE, just like ALTER TABLE, can be rolled back for any engine
* but don't auto-convert warnings into errors for engine warnings
  (handler::create) - this matches ALTER TABLE behavior
* and not when creating a default record, these errors are handled
  specially (and replaced with ER_INVALID_DEFAULT)
* always issue a Note when a non-unique key is truncated, because it's
  not a Warning that can be converted to an Error. Before this commit
  it was a Note for blobs and a Warning for all other data types.
2021-01-11 21:54:47 +01:00
Sergei Golubchik
6f707430e5 cleanup: copy RAII helpers from 10.5, cleanup test 2021-01-11 21:54:47 +01:00
Sergei Golubchik
4568a72ce4 don't do a warning for bad table options in replication slave thread
otherwise ALTER TABLE can break replication
2021-01-11 21:54:47 +01:00
Sergei Golubchik
674be2fd82 MDEV-18428 Memory: If transactional=0 is specified in CREATE TABLE, it is not possible to ALTER TABLE
fix "engine does not support TRANSACTIONAL=1" error message to
match user input
2021-01-11 21:54:47 +01:00
Sergei Golubchik
22b171d304 MDEV-17852 Altered connection limits for user have no effect
update mqh in struct user_conn after taking it from the cache
2021-01-11 21:54:47 +01:00
Sergei Golubchik
fc0d9a470c MDEV-22966 Server crashes or hangs with SET ROLE when started with skip-grant-tables 2021-01-11 21:54:47 +01:00