This patch addresses two problems with
rpl.rpl_seconds_behind_master_spike
First, --sync_slave_with_master / select master_pos_wait
seems to have a bug where it will hang after all master
events have been executed.
This patch removes the sync_slave_with_master command from
the test, where it not required anyway as it is used to
declare explicit cleanup
Second, the test uses timestamps to ensure that the
Seconds_Behind_Master value does not point to a time too
far in the past. The checks of these timestamps were
too strict, because they could be slightly inconsistent
with the master and the SBM would be counted as invalid
when it was actually correct.
To fix this, a slight buffer was added to the check
to ensure the value is valid but still does not point
too far in the past
Reviewed By:
===========
Andrei Elkin <andrei.elkin@mariadb.com>
Execution of the CREATE VIEW statement sent via binary protocol
where the flags of the COM_STMT_EXECUTE request a cursor to be opened
before running the statement results in an assert failure.
This assert fails since the data member thd->lex->result has not null
value pointing to an instance of the class Select_materialize.
The data member thd->lex->result is assigned a pointer to the class
Select_materialize in the function mysql_open_cursor() that invoked
in case the packet COM_STMT_EXECUTE requests a cursor to be opened.
After thd->lex->result is assigned a pointer to an instance of the
class Select_materialize the function mysql_create_view() is called
(indirectly via the function mysql_execute_statement()) and the assert
fails.
The assert
DBUG_ASSERT(!lex->proc_list.first && !lex->result &&
!lex->param_list.elements);
was added by the commit 591c06d4b7.
Unfortunately , the condition
!lex->result
was specified incorrect. It was supposed that the thd->lex->result
is set only by parser on handling the clauses SELECT ... INTO
but indeed it is also set inside mysql_open_cursor() and
that fact was missed by the assert's condition.
So, the fix for this issue is to just remove the condition
!lex->result
from the failing assert.
It's misleading to compare and write to user number of columns and fields.
Thus, it would be better to remove that check and let use see a subsequent
error message about missing or mispaced column.
row_import::match_schema(): remove misleading check
The InnoDB DATA DIRECTORY attribute is not implemented via
symbolic links but something similar, *.isl files that contain
the names of data files.
InnoDB failed to ignore the DATA DIRECTORY attribute even though
the server was started with --skip-symbolic-links.
Native ALTER TABLE in InnoDB will retain the DATA DIRECTORY attribute
of the table, no matter if the table will be rebuilt or not.
Generic ALTER TABLE (with ALGORITHM=COPY) as well as TRUNCATE TABLE
will discard the DATA DIRECTORY attribute.
All tests have been run with and without the ./mtr option
--mysqld=--skip-symbolic-links
and some tests that use the InnoDB DATA DIRECTORY attribute
have been adjusted for this.
Add a --rocksdb_ignore_datadic_errors plugin option for MyRocks.
The default is 0, and this means MyRocks will call abort() if it detects
a DDL mismatch.
Setting rocksdb_ignore_datadic_errors=1 makes MyRocks to try to ignore the
errors and allow to start the server for repairs.
In ha_rocksdb::open(), check if the number of indexes seen from the
SQL layer matches the number of indexes in the internal MyRocks data
dictionary.
Produce an error if there is a mismatch. (If we don't produce this error,
we are likely to crash as soon as we attempt to use an index)
MDEV-27107 prevent two mariadb-upgrade running in parallel
MDEV-27279 mariadb_upgrade add --check-if-upgrade-is-needed /
restrict tests to major version
Code is based of pull request from Daniel Black, but with a several
extensions.
- mysql_upgrade now locks the mysql_upgrade file with my_lock()
(Advisory record locking). This ensures that two mysql_upgrades
cannot be run in parallel.
- Added --check-if-upgrade-is-needed to mysql_upgrade. This will return
0 if one has to run mysql_upgrade.
Other changes:
- mysql_upgrade will now immediately exit if the major version and minor
version (two first numbers in the version string) is same as last run.
Before this change mysql_upgrade was run if the version string was different
from last run.
- Better messages when there is no need to run mysql_upgrade.
- mysql_upgrade --verbose now prints out a lot more information about
the version checking.
- mysql_upgrade --debug now uses default debug arguments if there is no
option to --debug
- "MySQL" is renamed to MariaDB in the messages
- mysql_upgrade version increased to 2.0
Notes
Verifying "prevent two mariadb-upgrade running in parallel" was
done in a debugger as it would be a bit complex to do that in mtr.
Reviewer: Danial Black <daniel@mariadb.org>
Database names are 64 utf8 characters per the system tables
that refer to them.
The current database() function is returning 34 characters.
The result of limiting this function results to max length of 34
became apparent when used in a UNION ALL where the results are
truncated to 34 characters.
For (uninvestigated) reasons, SELECT DATABASE() on its own
would always return the right number of characters.
Thanks Alexander Barkov for the review.
Thanks dave for noticing the bug in the stackexchange post
https://dba.stackexchange.com/questions/306183/why-is-my-database-name-truncated
Running a query using cursor could lead to a server crash on
building a temporary table used for handling the query.
For example, the following cursor
DECLARE cur1 CURSOR FOR
SELECT t2.c1 AS c1 FROM t1 LEFT JOIN t2 ON t1.c1 = t2.c1
WHERE EXISTS (SELECT 1 FROM t1 WHERE c2 = -1) ORDER BY c1;
declared and executed inside a stored routine could result in server
crash on creating a temporary table used for handling the ORDER BY clause.
Crash occurred on attempt to create the temporary table's fields based
on fields whose data located in a memory root that already freed.
It happens inside the function return_zero_rows() where the method
Select_materialize::send_result_set_metadata() is invoked for cursor case.
This method calls the st_select_lex_unit::get_column_types() in order to
get a list of items with types of columns for the temporary table being created.
The method st_select_lex_unit::get_column_types() returns
first_select()->join->fields
in case it is invoked for a cursor. Unfortunately, this memory has been already
deallocated bit earlier by calling
join->join_free();
inside the function return_zero_rows().
In case the query listed in the example is run in conventional way (without
using cursor) the method st_select_lex_unit::get_column_types()
returns first_select()->item_list that is not touched by invocation
of the method join->join_free() so everything is fine for that.
So, to fix the issue the resources allocated for the JOIN class should be
released after any activities with the JOIN class has been completed,
that is as the last statement before returning from the function
return_zero_rows().
This patch includes tests both for the case when a cursor is run explicitly
from within a stored routine and for the case when a cursor is opened
implicitly as prescribed by the STMT_ATTR_CURSOR_TYPE attribute of
binary protocol (the case of prepared statement).
.. to be the same as startup.
In resolving MDEV-27461, BUF_LRU_MIN_LEN (256) is the minimum number of
pages for the innodb buffer pool size. Obviously we need more than just
flushing pages. Taking the 16k page size and its default minimum, an
extra 25% is needed on top of the flushing pages to make a workable buffer
pool.
The minimum innodb_buffer_pool_chunk_size (1M) restricts the minimum
otherwise we'd have a pool made up of different chunk sizes.
The resulting minimum innodb buffer pool sizes are:
Page Size, Previously minimum (startup), with change.
4k 5M 2M
8k 5M 3M
16k 5M 5M
32k 24M 10M
64k 24M 20M
With this patch, SET GLOBAL innodb_buffer_pool_size minimums are
enforced.
The evident minimum system variable size for innodb_buffer_pool_size
is 2M, however this is only setable if using 4k page size. As
the order of the page_size and buffer_pool_size aren't fixed, we can't
hide this change.
Subsequent changes:
* innodb_buffer_pool_resize_with_chunks.test - raised of pool resize due to new
minimums. Chunk size also needed increase as the test was for
pool_size < chunk_size to generate a warning.
* Removed srv_buf_pool_min_size and replaced use with MYSQL_SYSVAR_NAME(buffer_pool_size).min_val
* Removed srv_buf_pool_def_size and replaced constant defination in
MYSQL_SYSVAR_LONGLONG(buffer_pool_size)
* Reordered ha_innodb to allow for direct use of MYSQL_SYSVAR_NAME(buffer_pool_size).min_val
* Moved buf_pool_size_align into ha_innodb to access to MYSQL_SYSVAR_NAME(buffer_pool_size).min_val
* loose-innodb_disable_resize_buffer_pool_debug is needed in the
innodb.restart.opt test so that under debug mode, resizing of the
innodb buffer pool can occur.
There were no InnoDB changes in the MySQL 5.7.37 release that would be
relevant to MariaDB Server. We will merely update the reported
InnoDB version number.
Range optimizer incorrectly was used for ENUM columns
when the operation collation did not match the column collation.
Adding a virtual implementation of Field_enum::can_optimize_range()
which tests if the column and the operation collation match.
mtr detects a forced combination if the command line for a test already
includes all options from this combination. options are stored in a perl
hash as (key,value) pairs.
this breaks if the command line has two options with the same name,
like --plugin-load-add=foo --plugin-load-add=bar, and the combination
forces plugin foo.
In particular, this resulted in warnings when running
federated.federatedx_versioning test
The column INFORMATION_SCHEMA.INNODB_LOCKS.LOCK_DATA
would report NULL when the page that contains the locked
record does not reside in the buffer pool.
Pages may be evicted from the buffer pool due to some background
activity, such as the purge of transaction history loading
undo log pages to the buffer pool. The regression tests intentionally
run with a small buffer pool size setting.
To prevent the intermittent test failures, we will filter out the
contents of the LOCK_DATA column from the output.
Fix regression introduced in MDEV-19893
Some errors must be sent with seqno = 0, e.g those that are detected
before server sends its first "welcome" packet (e.g too many connections)
This was not taken into account originally in MDEV-19893 fix.
We need to check sql_errno, before fixing sequence number, to see
if the error we send is really an out-of-bound, e.g a KILL.
In Galera case we call check_engine that could set create_info->db_type
to NULL e.g. if TEMPORARY is not supported by storage engine. Thus,
we need to restore it after that call because it is needed later
on mysql_create_table that will also call check_engine.
For MERGE-tables we need to init children list before calling
show_create_table and then detach children before we continue
normal mysql_create_like_table execution.
Re-execution of a query containing subquery in the FROM clause results
in assert failure in case the query is run as part of a stored routine or
as a prepared statement AND derived table merge optimization is off.
As an example, the following test case
CREATE TABLE t1 (a INT) ;
CREATE PROCEDURE sp() SELECT * FROM (SELECT a FROM t1) tb;
CALL sp();
SET optimizer_switch='derived_merge=off';
CALL sp();
results in assert failure on the second invocation of the 'sp' stored routine.
The reason for assertion failure is that the expression
derived->is_excluded()
returns the value true where the value false expected.
The method is_excluded() returns the value true for a derived table
that has been merged to a parent select. Such transformation happens as part
of Derived Table Merge Optimization that is performed on first invocation of
a stored routine or a prepared statement containing a query with subquery
in the FROM clause of the main SELECT.
When the same routine or prepared statement is run the second time and
Derived Table Merge Optimization is OFF the MariaDB server tries to materialize
a derived table specified by the subquery that fails since this subquery
has already been merged to the top-most SELECT. This transformation is permanent
and can't be reverted. That is the reason why the assert
DBUG_ASSERT(!derived->is_excluded());
fails inside the function TABLE_LIST::set_check_materialized().
Similar behaviour can be observed in case a stored routine or prepared statement
containing a SELECT statement with subquery in the FROM clause, first is run
with the optimizer_switch option set to derived_merge=off and re-run after this
option has been switched to derived_merge=on. In this case a derived table for
subquery is materialized on the first execution and marked as merged derived
table on the second execution that results in error with misleading error
message:
MariaDB [test]> CALL sp1();
ERROR 1030 (HY000): Got error 1 "Operation not permitted" from storage engine MEMORY
To fix the issue, a derived table that has been already optimized shouldn't be
re-marked for one more round of optimization.
One significant consequence following from suggested change is that the data
member TABLE_LIST::derived_type is not updated once the table optimization
has been done. This fact should be taken into account when Prepared Statement
being handled since once a table listed in a query has been optimized on
execution of the statement PREPARE FROM it won't be touched anymore on handling
the statement EXECUTE.
One side effect caused by this change could be observed for the following
test case:
CREATE TABLE t1 (s1 INT);
CREATE VIEW v1 AS
SELECT s1,s2 FROM (SELECT s1 as s2 FROM t1 WHERE s1 <100) x, t1 WHERE t1.s1=x.s2;
INSERT INTO v1 (s1) VALUES (-300);
PREPARE stmt FROM "INSERT INTO v1 (s1) VALUES (-300)";
EXECUTE stmt;
Execution of the above EXECUTE statement results in issuing the error
ER_COLUMNACCESS_DENIED_ERROR since table_ref->is_merged_derived() is false
and check_column_grant_in_table_ref() called for a temporary table that
shouldn't be. To fix this issue the function find_field_in_tables has been
modified in such a way that the function check_column_grant_in_table_ref()
is not called for a temporary table.
This bug could cause a crash of the server for queries with a derived table
whose specification contained the set function using a subquery over a view
as its only argument. The crash could happen if the specification of the
view contained an outer reference. In this case the aggregation select
could be determined incorrectly.
The crash also could be observed if a CTE is used instead of the view, but
only for queries having at least two references to the CTE.
The cause of this bug is the same as of the bug MDEV-24454.
This bug manifested itself at the second execution of the queries that
contained a set function whose only argument was outer reference to
a column of a mergeable view or derived table or CTE. The first execution
of such query worked fine, but the second execution of the query caused
a crash of the server because the aggregation select for the used set
function was determined incorrectly at the name resolution phase of the
second execution.
This patch reverts the fixes of the bugs MDEV-24454 and MDEV-25631 from
the commit 3690c549c6.
It leaves the changes in plugin/feedback/feedback.cc and corresponding
test files introduced in this commit intact.
Proper fixes for the bug MDEV-24454 and MDEV-25631 will follow immediately.
