- MDEV-11621 rpl.rpl_gtid_stop_start fails sporadically in buildbot
- MDEV-11620 rpl.rpl_upgrade_master_info fails sporadically in buildbot
The issue above was probably that the build machine was overworked and the
shutdown took longer than 30 resp 10 seconds, which caused MyISAM tables
to be marked as crashed.
Fixed by flushing myisam tables before doing a forced shutdown/kill.
I also increased timeout for forced shutdown from 10 seconds to 60 seconds
to fix other possible issues on slow machines.
Fixed also some compiler warnings
- privilege_table_io.test didn't properly reset roles_mapping
- Fixed memory allocation problem with CHECK CONSTRAINT, found when
running --valgrind main.check_constraint
- Atomic writes are enabled by default
- Automatically detect if device supports atomic write and use it if
atomic writes are enabled
- Remove ATOMIC WRITE options from CREATE TABLE
- Atomic write is a device option, not a table options as the table may
crash if the media changes
- Add support for SHANNON SSD cards
buf_flush_init_flush_rbt() was called too early in MariaDB server 10.0,
10.1, MySQL 5.5 and MySQL 5.6. The memory leak has been fixed in
the XtraDB storage engine and in MySQL 5.7.
As a result, when the server is started to initialize new data files,
the buf_pool->flush_rbt will be created unnecessarily and then leaked.
This memory leak was noticed in MariaDB server 10.1 when running the
test encryption.innodb_first_page.
Perform a slow shutdown at the start of the test, and create all
InnoDB tables with STATS_PERSISTENT=0, so that any I/O related to
background tasks (change buffer merge, purge, persistent statistics)
should be eliminated.
This should be a non-functional change. I was unable to repeat
MDEV-11626 innodb.innodb-change-buffer-recovery fails for xtradb
and cannot determine the reason for the failure without having access
to the files.
The repeatability of MDEV-11626 should not be affected by these changes.
Problem was with deleting non existing .frm file for a storage engine that
doesn't have .frm files (yet)
Fixed by not giving an error for non existing .frm files for storage engines
that are using discovery
Fixed also valgrind supression related to the given test case
Sometimes innodb_data_file_size_debug was reported as INT UNSIGNED
instead of BIGINT UNSIGNED. Make it uint instead of ulong to get
a more deterministic result.
for the default installation.
It is now defined as "C:/Program Files/MariaDB ${MYSQL_BASE_VERSION}"
which is where installer indeed puts it by default.
It still does not cover every case -32bit installer on 64 bit Windows would put installation
root under "C:/Program Files (x86)", but better than the path used
previously C:/MariaDB${MYSQL_BASE_VERSION}, which was never correct.
in default installation.
Added plugin-dir to the [client] section of the generated my.ini,
so that installed services (MSI or mysql_install_db.exe) would be able to
find plugin directory.
Cherry-pick: f4a0af070ce49abae60040f6f32e1074309c27fb
Author: Dmitry Lenev <dmitry.lenev@oracle.com>
Date: Mon Jul 25 16:06:52 2016 +0300
Fix for bug #16672723 "CAN'T FIND TEMPORARY TABLE".
Attempt to execute prepared CREATE TABLE SELECT statement which used
temporary table in the subquery in FROM clause and stored function
failed with unwarranted ER_NO_SUCH_TABLE error. The same happened
when such statement was used in stored procedure and this procedure
was re-executed.
The problem occurred because execution of such prepared statement/its
re-execution as part of stored procedure incorrectly set
Query_table_list::query_tables_own_last marker, indicating the last
table which is directly used by statement. As result temporary table
used in the subquery was treated as indirectly used/belonging to
prelocking list and was not pre-opened by open_temporary_tables()
call before statement execution. Thus causing ER_NO_SUCH_TABLE errors
since our code assumes that temporary tables need to be correctly
pre-opened before statement execution.
This problem became visible only in version 5.6 after patches related to
bug 11746602/27480 "EXTEND CREATE TEMPORARY TABLES PRIVILEGE TO ALLOW
TEMP TABLE OPERATIONS" since they have introduced pre-opening of temporary
tables for statements.
Incorrect setting of Query_table_list::query_tables_own_last happened
in LEX::first_lists_tables_same() method which is called by CREATE TABLE
SELECT implementation as part of LEX::unlink_first_table(), which temporary
excludes table list element for table being created from the query table
list before handling SELECT part.
LEX::first_lists_tables_same() tries to ensure that global table list of
the statement starts with the first table list element from the first
statement select. To do this it moves such table list element to the head
of the global table list. If this table happens to be last directly-used
table for the statement, query_tables_own_last marker is pointing to it.
Since this marker was not updated when table list element was moved we
ended up with all tables except the first table separated by it as if
they were not directly used by statement (i.e. belonged to prelocked
tables list).
This fix changes code of LEX::first_lists_tables_same() to update
query_tables_own_last marker in cases when it points to the table
being moved. It is set to the table which precedes table being moved
in this case.
Make the slave SQL thread always output to the error log the message "Slave
SQL thread exiting, replication stopped in ..." whenever it previously
outputted "Slave SQL thread initialized, starting replication ...".
Before this patch, it was somewhat inconsistent in which cases the message
would be output and in which not, depending on the exact time and cause of
the condition that caused the SQL thread to stop.
MTR raises default wait_for_pos_timeout from 300 to 1500 when tests
are run with valgrind. The same needs to be done for other
replication-related waits.
The change should fix one of failures mentioned in MDEV-10653
(rpl.rpl_parallel fails in buildbot with timeout), the one
on the valgrind builder; but not all of them
Memory was leaked when ALTER TABLE is attempted on a table
that contains corrupted indexes.
The memory leak was reported by AddressSanitizer for the test
innodb.innodb_corrupt_bit. The leak was introduced into
MariaDB Server 10.0.26, 10.1.15, 10.2.1 by the following:
commit c081c978a2
Merge: 1d21b22155a482e76e65
Author: Sergei Golubchik <serg@mariadb.org>
Date: Tue Jun 21 14:11:02 2016 +0200
Merge branch '5.5' into bb-10.0
This makes no functional change to MariaDB Server 10.2.
XtraDB is not being built, and in InnoDB, there was no
memory leak in buf_dblwr_process() in MariaDB Server 10.2.
Most of them are trivial, except for the thread_sync_t refactoring.
We must not invoke memset() on non-POD objects.
mtflush_work_initialized: Remove. Refer to mtflush_ctx != NULL instead.
thread_sync_t::thread_sync_t(): Refactored from
buf_mtflu_handler_init().
thread_sync_t::~thread_sync_t(): Refactored from
buf_mtflu_io_thread_exit().
MariaDB Server is unnecessarily evaluating the arguments of
DBUG_PRINT() macros when the label is not defined.
The macro DBUG_LOG() for C++ operator<< output which was added for
InnoDB diagnostics in MySQL 5.7 is missing from MariaDB. Unlike the
MySQL 5.7 implementation, MariaDB will avoid allocating and
initializing the output string when the label is not defined.
Introduce DBUG_OUT("crypt") and DBUG_OUT("checksum") for some InnoDB
diagnostics, replacing some use of ib::info().
In the backport of Bug#24450908 UNDO LOG EXISTS AFTER SLOW SHUTDOWN
from MySQL 5.7 to the MySQL 5.6 based MariaDB Server 10.1, we must
use a mutex when HAVE_ATOMIC_BUILTINS is not defined.
Also, correct a function comment. In MySQL 5.6 and MariaDB Server 10.1,
also temporary InnoDB tables are redo-logged.
1. The rows of a recursive CTE at some point may overflow
the HEAP temporary table containing them. At this point
the table is converted to a MyISAM temporary table and the
new added rows are placed into this MyISAM table.
A bug in the of select_union_recursive::send_data prevented
the server from writing the row that caused the overflow
into the temporary table used for the result of the iteration
steps. This could lead, in particular,to a premature end
of the iterations.
2. The method TABLE::insert_all_rows_into() that was used
to copy all rows of one temporary table into another
did not take into account that the destination temporary
table must be converted to a MyISAM table at some point.
This patch fixed this problem. It also renamed the method
into TABLE::insert_all_rows_into_tmp_table() and added
an extra parameter needed for the conversion.
encryption.innodb_scrub: Clean up. Make it also cover ROW_FORMAT=COMPRESSED,
removing the need for encryption.innodb_scrub_compressed.
Add a FIXME comment saying that we should create a secondary index, to
demonstrate that also undo log pages get scrubbed. Currently that is
not working!
Also clean up encryption.innodb_scrub_background, but keep it disabled,
because the background scrubbing does not work reliably.
Fix both tests so that if something is not scrubbed, the test will be
aborted, so that the data files will be preserved. Allow the tests to
run on Windows as well.
InnoDB shutdown failed to properly take fil_crypt_thread() into account.
The encryption threads were signalled to shut down together with other
non-critical tasks. This could be much too early in case of slow shutdown,
which could need minutes to complete the purge. Furthermore, InnoDB
failed to wait for the fil_crypt_thread() to actually exit before
proceeding to the final steps of shutdown, causing the race conditions.
Furthermore, the log_scrub_thread() was shut down way too early.
Also it should remain until the SRV_SHUTDOWN_FLUSH_PHASE.
fil_crypt_threads_end(): Remove. This would cause the threads to
be terminated way too early.
srv_buf_dump_thread_active, srv_dict_stats_thread_active,
lock_sys->timeout_thread_active, log_scrub_thread_active,
srv_monitor_active, srv_error_monitor_active: Remove a race condition
between startup and shutdown, by setting these in the startup thread
that creates threads, not in each created thread. In this way, once the
flag is cleared, it will remain cleared during shutdown.
srv_n_fil_crypt_threads_started, fil_crypt_threads_event: Declare in
global rather than static scope.
log_scrub_event, srv_log_scrub_thread_active, log_scrub_thread():
Declare in static rather than global scope. Let these be created by
log_init() and freed by log_shutdown().
rotate_thread_t::should_shutdown(): Do not shut down before the
SRV_SHUTDOWN_FLUSH_PHASE.
srv_any_background_threads_are_active(): Remove. These checks now
exist in logs_empty_and_mark_files_at_shutdown().
logs_empty_and_mark_files_at_shutdown(): Shut down the threads in
the proper order. Keep fil_crypt_thread() and log_scrub_thread() alive
until SRV_SHUTDOWN_FLUSH_PHASE, and check that they actually terminate.
Port a bug fix from MySQL 5.7, so that all undo log pages will be freed
during a slow shutdown. We cannot scrub pages that are left allocated.
commit 173e171c6fb55f064eea278c76fbb28e2b1c757b
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date: Fri Sep 9 18:01:27 2016 +0530
Bug #24450908 UNDO LOG EXISTS AFTER SLOW SHUTDOWN
Problem:
========
1) cached undo segment is not removed from rollback segment history
(RSEG_HISTORY) during slow shutdown. In other words, If the segment is
not completely free, we are failing to remove an entry from the history
list. While starting the server, we traverse all rollback segment slots
history list and make it as list of undo logs to be purged in purge
queue.
In that case, purge queue will never be empty after slow shutdown.
2) Freeing of undo log segment is linked with removing undo log header
from history.
Fix:
====
1) Have separate logic of removing the undo log header from
history list from rollback segment slots and remove it from
rollback segment history even though it is not completely free.
Reviewed-by: Debarun Banerjee <debarun.banerjee@oracle.com>
Reviewed-by: Marko Mäkelä <marko.makela@oracle.com>
RB:13672
MariaDB Server 10.0.28 and 10.1.19 merged code from Percona XtraDB
that introduced support for compressed columns. Much but not all
of this code was disabled by placing #ifdef HAVE_PERCONA_COMPRESSED_COLUMNS
around it.
Among the unused but not disabled code is code to access
some new system tables related to compressed columns.
The creation of these system tables SYS_ZIP_DICT and SYS_ZIP_DICT_COLS
would cause a crash in --innodb-read-only mode when upgrading
from an earlier version to 10.0.28 or 10.1.19.
Let us remove all the dead code related to compressed columns.
Users who already upgraded to 10.0.28 and 10.1.19 will have the two
above mentioned empty tables in their InnoDB system tablespace.
Subsequent versions of MariaDB Server will completely ignore those tables.
10.1 is merged into 10.2 now. Two issues are left to fix:
(1) encryption.innochecksum test
(2) read_page0 vs page_0_crypt_read
(1) innochecksum tool did not compile after merge because
buf_page_is_corrupted uses fil_crypt_t that has been changed.
extra/CMakeLists.txt: Added fil/fil0crypt.cc as dependency
as we need to use fil_crypt_verify_checksum for encrypted pages.
innochecksum.cc: If we think page is encrypted i.e.
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION != 0 we call
fil_crypt_verify_checksum() function to compare calculated
checksum to stored checksum calculated after encryption
(this is stored on different offset i.e.
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION + 4).
If checksum does not match we call normal buf_page_is_corrupted
to compare calculated checksum to stored checksum.
fil0crypt.cc: add #ifdef UNIV_INNOCHECKSUM to be able to compile
this file for innochecksum tool.
(2) read_page0 is not needed and thus removed.
after aborted InnoDB startup
This bug was repeatable by starting MariaDB 10.2 with an
invalid option, such as --innodb-flush-method=foo.
It is not repeatable in MariaDB 10.1 in the same way, but the
problem exists already there.
after aborted InnoDB startup
This bug was repeatable by starting MariaDB 10.2 with an
invalid option, such as --innodb-flush-method=foo.
It is not repeatable in MariaDB 10.1 in the same way, but the
problem exists already there.
