Commit graph

192575 commits

Author SHA1 Message Date
Marko Mäkelä
b81382887c MDEV-25512 Deadlock between sux_lock::u_x_upgrade() and sux_lock::u_lock()
In the SUX_LOCK_GENERIC implementation, we can remember at most
one pending exclusive lock request. If multiple exclusive lock
requests are pending, the WRITER_WAITING flag will be cleared when
the first waiting writer acquires the exclusive lock.

ssux_lock_low::update_lock(): If WRITER_WAITING is set, wake up
the writer even if the UPDATER flag is set, because the waiting
writer may be in the process of upgrading its U lock to X.

rw_lock::read_unlock(): Also indicate that an X lock waiter must
be woken up if an U lock exists.

This fix may cause unnecessary wake-ups and system calls, but this
is the best that we can do. Ideally we would use the MDEV-25404
idea of a separate 'writer' mutex, but there is no portable way to
request that a non-recursive mutex be created, and InnoDB requires
the ability to transfer buf_block_t::lock ownership to an I/O thread.

To allow problems like this to be caught more reliably in the future,
we add a unit test for srw_mutex, srw_lock, ssux_lock, sux_lock.
2021-04-25 12:58:16 +03:00
Marko Mäkelä
1a647b700f MDEV-25487 Assertion failed in lock_rec_move
row_ins_clust_index_entry_low(): Do not enable bulk insert if
any record locks exist on the table. Bulk insert is assumed to
be covered only by an exclusive table lock, with no row-level
locking or undo logging.
2021-04-23 10:07:08 +03:00
Sujatha
ee20e26dbb MDEV-20220 post push fix
Remove garbage files.
2021-04-23 11:58:39 +05:30
Marko Mäkelä
bfc0110b9b Null-merge the revert of MDEV-24589
While MDEV-24589 causes the regression
MDEV-25491 Race condition between DROP TABLE and purge of
SYS_INDEXES record
we must not revert it from 10.6, because MDEV-24589 is a prerequisite for
MDEV-25180 Atomic ALTER TABLE
that is targeting 10.6. The regression will be dealt with later.
2021-04-22 18:39:40 +03:00
Marko Mäkelä
18f35a8608 Merge 10.5 into 10.6 2021-04-22 18:13:47 +03:00
Marko Mäkelä
8121d03fd4 MDEV-25404 fixup: Fix ssux_lock_low::u_wr_upgrade()
The U-to-X upgrade turned out to be incorrect. A debug assertion
failed in wr_wait(), called from mtr_defer_drop_ahi() in a stress
test with innodb_adaptive_hash_index=ON.

A correct upgrade procedure ought to be readers.fetch_add(WRITER-1)
to register ourselves as a WRITER (or waiting writer) and to release
the reference that was being held for the U lock.

Thanks to Matthias Leich for catching the problem.
2021-04-22 15:43:30 +03:00
Marko Mäkelä
1636db5443 Revert "MDEV-24589 DROP TABLE is not crash-safe"
This reverts commit e731a28394.

A crash occurred during the test stress.ddl_innodb when
fil_delete_tablespace() for DROP TABLE was waiting in
fil_check_pending_operations() and a purge thread for handling
an earlier DROP INDEX was attempting to load the index root page
in btr_free_if_exists() and btr_free_root_check(). The function
buf_page_get_gen() would write out several times
"trying to read...being-dropped tablespace"
before giving up and committing suicide.

It turns out that during any page access in btr_free_if_exists(),
fil_space_t::set_stopping() could have been invoked by
fil_check_pending_operations(), as part of dropping the tablespace.
Preventing this race condition would require extensive changes
to the allocation code or some locking mechanism that would ensure
that we only set the flag if btr_free_if_exists() is not in progress.

Either way, that could be a too risky change in a GA release.
Because MDEV-24589 is not strictly necessary in the 10.5 release
series and it only is a requirement for MDEV-25180 in a later
major release, we will revert the change from 10.5.
2021-04-22 14:57:23 +03:00
sachin
21973d0d8e MDEV-22953 main.flush_read_lock failed in buildbot with XAER_NOTA: Unknown XID
Issue:- Since there is no waiting for the actual disconnection of the con_tmp
(which does XA prepare of test1), We can have a issue when test1 is not
prepared and we are calling rollback on test1 , giving XAER_NOTA: Unknown XID
error

Solution:- Wait for the complete disconnection of con_tmp
2021-04-22 10:37:33 +01:00
Marko Mäkelä
54c460ace6 Merge 10.5 into 10.6 2021-04-22 08:43:03 +03:00
Marko Mäkelä
b728c3dbd9 fixup dd6ad38068: remove dead code 2021-04-22 08:42:55 +03:00
Marko Mäkelä
df33b719ca Merge 10.4 into 10.5 2021-04-22 08:25:40 +03:00
Marko Mäkelä
cbbca7edf6 MDEV-25483: Shutdown crash during innodb.innodb_buffer_pool_resize_temporary
buf_resize_callback(): Correct an invalid assertion, and enable
the assertion in debug builds only.

Between buf_resize_start() and buf_resize_shutdown(),
srv_shutdown_state must be less than SRV_SHUTDOWN_CLEANUP.

The incorrect assertion had been introduced in
commit 5e62b6a5e0 (MDEV-16264).
As a result, the server could crash if shutdown was initiated
concurrently with initiating a change of innodb_buffer_pool_size.
2021-04-22 08:13:57 +03:00
Marko Mäkelä
ee455e6f2e Merge 10.3 into 10.4 2021-04-22 07:51:33 +03:00
Marko Mäkelä
0d267f7caa MDEV-25362 after-merge fix: Remove unnecessary code 2021-04-22 07:36:04 +03:00
Marko Mäkelä
6f271302b6 Merge 10.2 into 10.3 2021-04-22 07:32:51 +03:00
Thirunarayanan Balathandayuthapani
fb96ac0a49 MDEV-25474 Background thread returns uninitialized statistics
to mysql interpreter

InnoDB returns uninitialized statistics to mysql interpreter
when background thread is opening the table. So it leads to
assertion failure. In that case, InnoDB avoid sending
innodb statistics information to mysql interpreter.
2021-04-21 21:33:29 +05:30
Eugene Kosov
64eeb250eb MDEV-25457 Server crashes in row_undo_mod_clust_low upon rollback of read-only transaction
node->index was NULL.
But it's possible to get dict_table_t* from another source.
2021-04-21 17:42:16 +03:00
Andrei Elkin
2a7dd64425 MDEV-24526 binlog rotate via FLUSH LOGS may obsolate binlog file for recovery too eary
There was race between a committing transaction and the following in binlog
order FLUSH LOGS that could create a 2nd Binlog checkpoint (BCP) event
in the new file *before* the first logged-in-old-binlog transaction gets committed in
Innodb. That would cause the transaction loss at recovery, should
the server stop right after the BCP.

The race is tackled by enforcing the necessary set of mutexes to be acquired
by FLUSH-LOGS handler in the correct order (of the group commit leader
pattern).

Note, there remain two cases where a similar race is still possible:
  - the above race as it is when the server is run with ("unlikely")
    non-default `--binlog-optimize-thread-scheduling=0` (MDEV-24530), and
  - at unlikely event of bin-logging of Incident event (MDEV-24531) that
    also triggers binlog rotation,
    in both cases though with lesser chances after the current fixes.
2021-04-21 15:39:32 +03:00
Vicențiu Ciorbaru
bc04ded235 Follow-up post JSON_TABLE merge, table_join_options is not used 2021-04-21 15:17:39 +03:00
Vicențiu Ciorbaru
3019d1cb43 Update columnstore to following cleanup of SELECT_LEX 2021-04-21 14:51:15 +03:00
Vicențiu Ciorbaru
aeccdddedd MDEV-25441 WITH TIES is not respected with SQL_BUFFER_RESULT and constant in ORDER BY
Pushing LIMIT to temp aggregation table is possible, but not when WITH
TIES is used. In a degenerate case with constant ORDER BY, the constant
gets removed and the code assumed the limit is push-able.

Ensure that if WITH TIES is present, that this does not happen.
2021-04-21 14:09:17 +03:00
Vicențiu Ciorbaru
44a6af65f5 MDEV-25430: ROW | ROWS should be a required keyword after OFFSET start
Implemented according to standard OFFSET start { ROW | ROWS},
ROW and ROWS is mandatory after OFFSET.
2021-04-21 14:09:17 +03:00
Vicențiu Ciorbaru
299b935320 MDEV-23908: Implement SELECT ... OFFSET ... FETCH ...
This commit implements the standard SQL extension
OFFSET start { ROW | ROWS }
[FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } { ONLY | WITH TIES }]

To achieve this a reserved keyword OFFSET is introduced.

The general logic for WITH TIES implies:
1. The number of rows a query returns is no longer known during optimize
   phase. Adjust optimizations to no longer consider this.
2. During end_send make use of an "order Cached_item"to compare if the
   ORDER BY columns changed. Keep returning rows until there is a
   change. This happens only after we reached the row limit.
3. Within end_send_group, the order by clause was eliminated. It is
   still possible to keep the optimization of using end_send_group for
   producing the final result set.
2021-04-21 14:09:14 +03:00
Vicențiu Ciorbaru
2d595319bf cleanup: Select_limit_counters rename set_unlimited to clear
The function was originally introduced by eb0804ef5e
MDEV-18553: MDEV-16327 pre-requisits part 1: isolation of LIMIT/OFFSET handling

set_unlimited had an overloaded notion of both clearing the offset value
and the limit value. The code is used for SQL_CALC_ROWS option to
disable the limit clause after the limit is reached, while at the same
time the calling code suppreses sending of rows.

Proposed solution:
Dedicated clear method for query initialization (to ensure no garbage
remains between executions).
Dedicated set_unlimited that only alters the limit value.
2021-04-21 14:08:58 +03:00
Vicențiu Ciorbaru
3fcc4f6fc2 cleanup: make_aggr_tables_info pass parameters from existing local variables
Local variables such as keep_row_order are semantically clearer on the
intent, rather than using an or condition twice.
2021-04-21 14:08:58 +03:00
Vicențiu Ciorbaru
7e7ca09ce1 cleanup: test_if_subpart documented to highlight side effect
Also document (through function parameters names) which ORDER must be passed
to the function: first GROUP BY, then ORDER BY.
2021-04-21 14:08:58 +03:00
Vicențiu Ciorbaru
d028fb9ffa cleanup: reduce indentation of end_send
Reorder end_send to exit via return instead of large if-else block.
2021-04-21 14:08:58 +03:00
Vicențiu Ciorbaru
d624464b50 cleanup: remove table_options and corresponding accessor function
The variable is never used and only takes up SELECT_LEX space.
2021-04-21 14:08:58 +03:00
Vicențiu Ciorbaru
5e72099b83 cleanup: Delete unused function Select_limit_counters::is_unrestricted 2021-04-21 14:08:58 +03:00
Vicențiu Ciorbaru
13cf8f5e9a cleanup: Refactor select_limit in select lex
Replace
  * select_lex::offset_limit
  * select_lex::select_limit
  * select_lex::explicit_limit
with select_lex::Lex_select_limit

The Lex_select_limit already existed with the same elements and was used in
by the yacc parser.

This commit is in preparation for FETCH FIRST implementation, as it
simplifies a lot of the code.

Additionally, the parser is simplified by making use of the stack to
return Lex_select_limit objects.

Cleanup of init_query() too. Removes explicit_limit= 0 as it's done a bit later
in init_select() with limit_params.empty()
2021-04-21 14:08:58 +03:00
Marko Mäkelä
9abc6fd73f Cleanup: constexpr PFS_table_context::m_word_size 2021-04-21 12:44:23 +03:00
Marko Mäkelä
b59d07624c WITH_UBSAN: shift is too large for 32-bit int
As suggested by Vladislav Vaintroub, we must shift a constant of
unsigned long, not int.
2021-04-21 12:34:54 +03:00
Sergei Petrunia
dd6ad38068 Code cleanup: merge walk_items_for_table_list with walk_table_functions_for_list 2021-04-21 12:32:58 +03:00
Marko Mäkelä
4930f9c94b Merge 10.5 into 10.6 2021-04-21 11:45:00 +03:00
Vicențiu Ciorbaru
ef2749c90d Fixup: Event_queue_element_for_exec initializer list not supported on gcc-4.1 2021-04-21 11:21:57 +03:00
Marko Mäkelä
d104fe6f73 Merge 10.4 into 10.5 2021-04-21 10:27:13 +03:00
Marko Mäkelä
ceed768eea MDEV-25362 after-merge fix: GCC -Og -Wmaybe-uninitialized 2021-04-21 10:06:31 +03:00
Alexey Botchkov
502b769561 MDEV-17399 JSON_TABLE.
Aftermerge fixes.
2021-04-21 10:21:48 +04:00
Alexey Botchkov
41e368f22d MDEV-25149 JSON_TABLE: Inconsistency in implicit data type conversion.
Only return the error if field->store produced errors, not warnings.
2021-04-21 10:21:48 +04:00
Sergei Petrunia
0a09525625 MDEV-25202: JSON_TABLE: Early table reference leads to unexpected result set
Followup part#2: allocate the List object on the right mem-root, too.
2021-04-21 10:21:48 +04:00
Sergei Petrunia
e4665f417b MDEV-25202: JSON_TABLE: Early table reference leads to unexpected result set
Followup to review input address commit: pass the new parameter type
to find_field_in_table_ref().
2021-04-21 10:21:48 +04:00
Alexey Botchkov
a4353c25ca MDEV-25420 JSON_TABLE: ASAN heap-buffer-overflow in Protocol::net_store_data or consequent failur es.
fixed changed.
2021-04-21 10:21:48 +04:00
Sergei Petrunia
b0817ff8de MDEV-25202: JSON_TABLE: Early table reference leads to unexpected result set
Address review input: switch Name_resolution_context::ignored_tables from
table_map to a list of TABLE_LIST objects. The rationale is that table
bits may be changed due to query rewrites, etc, which may potentially
require updating ignored_tables.
2021-04-21 10:21:48 +04:00
Alexey Botchkov
0984b8ed08 MDEV-25420 JSON_TABLE: ASAN heap-buffer-overflow in Protocol::net_store_data or consequent failures.
error in the patch fixed.
2021-04-21 10:21:48 +04:00
Alexey Botchkov
91cd3c8f5b MDEV-25353 JSON_TABLE: Illegal mix of collations upon executing PS once, or SP/function twice.
Do not adapt to the argument charset anymore. Just use the utf8mb4 as
the default cherset.
2021-04-21 10:21:47 +04:00
Alexey Botchkov
59f3399e29 MDEV-25420 JSON_TABLE: ASAN heap-buffer-overflow in Protocol::net_store_data or consequent failures.
Create_tmp_table::add_field didn't consider BIT type field for
null_counter.
2021-04-21 10:21:47 +04:00
Alexey Botchkov
277aa532f3 MDEV-25408 JSON_TABLE: AddressSanitizer CHECK failed in Binary_string::realloc_raw
We need to update default values if we change the column's charsets.
2021-04-21 10:21:47 +04:00
Alexey Botchkov
991bfebe8f MDEV-25379 JSON_TABLE: ERROR ON clauses are ignored if a column is not on select list.
If a field is not in the read set - read it in the local buffer anyway to check for errors.
2021-04-21 10:21:47 +04:00
Sergei Petrunia
1eda21be1f Code cleanup: thd->lex->current_select->context == s_lex here, so use s_lex 2021-04-21 10:21:47 +04:00
Sergei Petrunia
a96408092c MDEV-25397: JSON_TABLE: Unexpected ER_MIX_OF_GROUP_FUNC_AND_FIELDS
When doing name resolution, do the same what WHERE/ON clauses do:
they don't count in select_lex->non_agg_field_used().
2021-04-21 10:21:47 +04:00