with window functions (mdev-10855).
This patch just modified the function pushdown_cond_for_derived()
to support this feature.
Some test cases demonstrating this optimization were added to
derived_cond_pushdown.test.
If the server is upgraded from a database that was created
before MDEV-12288, and if the undo logs in the database contain
an incomplete transaction that performed an INSERT operation,
the server would crash when rolling back that transaction.
trx_commit_low(): Relax a too strict transaction. This function
will also be called after completing the rollback of a recovered
transaction.
trx_purge_add_undo_to_history(): Merged from the functions
trx_purge_add_update_undo_to_history() and trx_undo_update_cleanup(),
which are removed. Remove the parameter undo_page, and instead call
trx_undo_set_state_at_finish() to obtain it.
trx_write_serialisation_history(): Treat undo and old_insert equally.
That is, after the rollback (or XA COMMIT) of a recovered transaction
before upgrade, move all logs (both insert_undo and update_undo) to
the purge queue.
JSON_EXTRACT behaves specifically in the comparison,
so we have to implement specific method for that in
Arg_comparator.
Conflicts:
sql/item_cmpfunc.cc
"Optimization for equi-joins of derived tables with GROUP BY"
should be considered rather as a 'proof of concept'.
The task itself is targeted at an optimization that employs re-writing
equi-joins with grouping derived tables / views into lateral
derived tables. Here's an example of such transformation:
select t1.a,t.max,t.min
from t1 [left] join
(select a, max(t2.b) max, min(t2.b) min from t2
group by t2.a) as t
on t1.a=t.a;
=>
select t1.a,tl.max,tl.min
from t1 [left] join
lateral (select a, max(t2.b) max, min(t2.b) min from t2
where t1.a=t2.a) as t
on 1=1;
The transformation pushes the equi-join condition t1.a=t.a into the
derived table making it dependent on table t1. It means that for
every row from t1 a new derived table must be filled out. However
the size of any of these derived tables is just a fraction of the
original derived table t. One could say that transformation 'splits'
the rows used for the GROUP BY operation into separate groups
performing aggregation for a group only in the case when there is
a match for the current row of t1.
Apparently the transformation may produce a query with a better
performance only in the case when
- the GROUP BY list refers only to fields returned by the derived table
- there is an index I on one of the tables T used in FROM list of
the specification of the derived table whose prefix covers the
the fields from the proper beginning of the GROUP BY list or
fields that are equal to those fields.
Whether the result of the re-writing can be executed faster depends
on many factors:
- the size of the original derived table
- the size of the table T
- whether the index I is clustering for table T
- whether the index I fully covers the GROUP BY list.
This patch only tries to improve the chosen execution plan using
this transformation. It tries to do it only when the chosen
plan reaches the derived table by a key whose prefix covers
all the fields of the derived table produced by the fields of
the table T from the GROUP BY list.
The code of the patch does not evaluates the cost of the improved
plan. If certain conditions are met the transformation is applied.
buf_page_io_complete(): Do not test bpage for NULL, because
it is declared (and always passed) as nonnull.
buf_flush_batch(): Remove the constant local variable count=0.
fil_ibd_load(): Use magic comment to suppress -Wimplicit-fallthrough.
ut_stage_alter_t::inc(ulint): Disable references to an unused parameter.
lock_queue_validate(), sync_array_find_thread(), rbt_check_ordering():
Define only in debug builds.
When undo log pages pre-exist from an upgrade, the
TRX_UNDO_PAGE_TYPE could be TRX_UNDO_INSERT==1 (for insert_undo)
TRX_UNDO_UPDATE==2 (for update_undo), instead of the new unified
page type 0 that was introduced in MDEV-12288.
The undo log page type does not really matter much, because the
undo log record type identifies the records independently
of the page type. So, the debug assertions can simply allow any
potential value of the TRX_UNDO_PAGE_TYPE (0, 1, or 2).
At shutdown, we would have lsn == srv_start_lsn == 0 if the server
was started up in --innodb-read-only mode with an old-format redo log.
This regression was caused by MDEV-13430, which skips some redo log
processing at startup when the redo log file format differs (and the
redo log has been determined to be logically empty).
Even though the MDEV-13430 change was introduced in MariaDB 10.2.8,
the MariaDB Server 10.2 series is unaffected by this, because
it will refuse to start up from a version-tagged redo log that is
not tagged to be in the MySQL 5.7.9 or MariaDB 10.2.2 format.
Starting with MariaDB 10.3.1, there are multiple version-tagged
redo log formats.
recv_recovery_from_checkpoint_start(): When skipping an empty
different-format redo log, initialize srv_start_lsn and
recv_sys->recovered_lsn from the redo log file.
The bug is result adding ability to have derived tables inside views.
Fixed checks should be a switch between view/derived or select derived and information schema.
Only a relevant subset of the InnoDB changes was merged.
In particular, two follow-up bug fixes for the bugs that
were introduced in 5.7.18 but not MariaDB 10.2.7 were omitted.
Because MariaDB 10.2.7 omitted the risky change
Bug#23481444 OPTIMISER CALL ROW_SEARCH_MVCC() AND READ THE INDEX
APPLIED BY UNCOMMITTED ROWS
we do not need the follow-up fixes that were introduced in
MySQL 5.6.37 and MySQL 5.7.19:
Bug#25175249 ASSERTION: (TEMPL->IS_VIRTUAL && !FIELD) || ...
Bug#25793677 INNODB: FAILING ASSERTION: CLUST_TEMPL_FOR_SEC || LEN
Analysis:
=========
During alter table rebuild, InnoDB fails to apply concurrent insert log.
If the insert log record is present across the blocks then apply phase
trying to access the next block without fetching it.
Fix:
====
During virtual column parsing, check whether the record is present
across the blocks before accessing the virtual column information.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 16243
Analysis:
========
During alter table rebuild, InnoDB fails to apply concurrent delete log.
Parsing and validation of merge record happens while applying the
log operation on a table. Validation goes wrong for the virtual column.
Validation assumes that virtual column information can't be the end
of the merge record end.
Fix:
====
Virtual column information in the merge record can be end of the merge
record. Virtual column information is written at the end for
row_log_table_delete().
Reviewed-by: Satya Bodapati<satya.bodapati@oracle.com>
RB: 16155
The test is for a bug that was introduced in MySQL 5.7.18
but not MariaDB 10.2, because MariaDB did not merge the change
that was considered incomplete and too risky for a GA release:
Bug#23481444 OPTIMISER CALL ROW_SEARCH_MVCC() AND READ THE INDEX
APPLIED BY UNCOMMITTED ROWS
So, we are only merging the test changes from the bug fix in MySQL 5.7.19,
not any code changes:
commit 4f86aca37d551cc756d9187ec901f8c4a68a0543
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date: Wed Apr 26 11:10:41 2017 +0530
Bug #25793677 INNODB: FAILING ASSERTION: CLUST_TEMPL_FOR_SEC || LEN
MariaDB 10.2 never contained the Oracle change
Bug#23481444 OPTIMISER CALL ROW_SEARCH_MVCC() AND READ THE
INDEX APPLIED BY UNCOMMITTED ROWS
because it was considered risky for a GA release and incomplete.
Remove the references that were added when merging MySQL 5.6.36
to MariaDB 10.0.31, 10.1.24, and 10.2.7.
Analysis:
========
(1) During TRUNCATE of file_per_table tablespace, dict_operation_lock is
released before eviction of dirty pages of a tablespace from the buffer
pool. After eviction, we try to re-acquire
dict_operation_lock (higher level latch) but we already hold lower
level latch (index->lock). This causes latch order violation
(2) Deadlock issue is present if child table is being truncated and it
holds index lock. At the same time, cascade dml happens and it took
dict_operation_lock and waiting for index lock.
Fix:
====
1) Release the indexes lock before releasing the dict operation lock.
2) Ignore the cascading dml operation on the parent table, for the
cascading foreign key, if the child table is truncated or if it is
in the process of being truncated.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
Reviewed-by: Kevin Lewis <kevin.lewis@oracle.com>
RB: 16122
Issue
====
The issue is that the info message that InnoDB prints when a table
is created with a reference which doesn't exist fills up the log as
it's printed for every insert when foreign_key_checks is disabled.
Fix
===
The fix is to display the message only if foreign_key_checks is
enabled.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
Problem:
=======
Offsets allocates memory from row_heap even for deleted row
traversal during table rebuild.
Solution:
=========
Empty the row_heap even for deleted record. So that
offsets don't allocate memory everytime.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 15694
Cherry-pick the commit from MySQL 5.7.19, and adapt the test case:
commit 45c933ac19c73a3e9c756a87ee1ba18ba1ac564c
Author: Aakanksha Verma <aakanksha.verma@oracle.com>
Date: Tue Mar 21 10:31:43 2017 +0530
Bug #25189192 ERRORS WHEN RESTARTING MYSQL AFTER RENAME TABLE.
PROBLEM
While renaming table innodb doesn't update the InnoDB Dictionary table
INNODB_SYS_DATAFILES incase there is change in database while doing
rename table. Hence on a restart the server log shows error that it
couldnt find table with old path before rename which has actually been
renamed. So the errors would only vanish if we update the system
tablespace
FIX
Update the innodb dictionary table with new path in the case there is
not a change in the table but the database holding the table as well.
Reviewed-by: Jimmy Yang<Jimmy.Yang@oracle.com>
RB: 15751
row_log_table_apply_convert_mrec(), row_log_table_apply_insert_low(),
row_log_table_apply_insert(), row_log_table_apply_update(): Remove
the trx_id parameter. The clustered index may contain DB_TRX_ID=0,
but PAGE_MAX_TRX_ID=0 is invalid for secondary indexes. Use the
current transaction ID when inserting secondary index records.
row_log_table_apply_op(): Remove the trx_id_col parameter.
This happens because on line 59 of /usr/bin/wsrep_sst_mysqldump
we have a check :
"if ! $MYSQL_CLIENT --version | grep 'Distrib 10.1' >/dev/null"
but the client is 10.2 so it doesnt match the grep expression.
Fixed check to be:
"if ! $MYSQL_CLIENT --version | grep 'Distrib 10.' >/dev/null"
so that every 10. version is accepted.
Disable change buffering, so that some data that was previously
written to the encrypted redo log will not end up being copied
to the unencrypted redo log due to change buffer merge.
The thd_destructor_proxy detects that no transactions are active and
starts srv_shutdown_bg_undo_sources(), but fails to take into account
that new transactions can still start, especially be slave but also
by other threads. In addition there is no mutex when checking for
active transaction so this is not safe.
We relax the failing InnoDB debug assertion by allowing the execution
of user transactions after the purge thread has been shut down.
FIXME: If innodb_fast_shutdown=0, we should somehow guarantee that no
new transactions can start after thd_destructor_proxy observed that
trx_sys_any_active_transactions() did not hold.
row_update_for_mysql(): Remove the wrapper function and
rename the function from row_update_for_mysql_using_upd_graph().
Remove the unused parameter mysql_rec.
This is basically a duplicate or a reincarnation of MDEV-117.
For some reason, the test innodb.mdev-117 started failing in 10.2.
It is uncertain when this test started failing. The test is
nondeterministic, because there is a race condition between the
concurrently executing DELETE IGNORE and DELETE statements.
When a deadlock is reported for DELETE IGNORE, the SQL layer would
call handler::print_error() but then proceed to the next row,
as if no error had happened (which is the purpose of DELETE IGNORE).
So, when it proceeded to handler::ha_rnd_next(), InnoDB would hit an
assertion failure, because the transaction no longer exists, and we
are not executing at the start of a statement.
handler::print_error(): If thd_mark_transaction_to_rollback(thd, true)
was called, clear the ME_JUST_WARNING and ME_JUST_INFO errflags, so
that a note or warning will be promoted to an error if the transaction
was aborted by a storage engine.
The file wait_innodb_all_purged.inc waited for InnoDB purge in a way
that only worked in debug builds. The file wait_all_purged.inc
provides a better mechanism.
If the latest InnoDB redo log checkpoint was stored in the
first checkpoint slot and not the second one, InnoDB would
incorrectly set log_sys->log.lsn to the previous checkpoint.
It is possible that this logic error did not exist before
commit 86927cc712, which
removed traces of multiple InnoDB redo logs, to prepare for
MDEV-12548 (Mariabackup for MariaDB 10.2). In the worst case,
this error could mean that InnoDB unnecessarily fails to
recover from redo log when the last-but-one checkpoint was
overwritten, but the last checkpoint is intact.
recv_find_max_checkpoint(), recv_find_max_checkpoint_0():
Do not overwrite the fields of log_sys->log with the information
of an older checkpoint.
recv_find_max_checkpoint(): Do not return DB_SUCCESS on an error.
recv_recovery_from_checkpoint_start(): Return early if the log is
in a version-tagged format but not in the latest format. (In this case,
the log must be logically empty, and there is nothing to apply.)
