There is only one purge_sys. Allocate it statically in order to avoid
dereferencing a pointer whenever accessing it. Also, align some
members to their own cache line in order to avoid false sharing.
purge_sys_t::create(): The deferred constructor.
purge_sys_t::close(): The early destructor.
undo::Truncate::create(): The deferred constructor.
Because purge_sys.undo_trunc is constructed before the start-up
parameters are parsed, the normal constructor would copy a
wrong value of srv_purge_rseg_truncate_frequency.
TrxUndoRsegsIterator: Do not forward-declare an inline constructor,
because the static construction of purge_sys.rseg_iter would not have
access to it.
trx_purge(): Remove the parameter limit or batch_size, which is
always passed as srv_purge_batch_size.
trx_purge_attach_undo_recs(): Remove the parameters purge_sys, batch_size.
Refer to srv_purge_batch_size.
trx_purge_wait_for_workers_to_complete(): Remove the parameter purge_sys.
Add innodb debug system variable, innodb_buffer_pool_load_pages_abort, to test
the behaviour of innodb_buffer_pool_load_incomplete.
(innodb_buufer_pool_dump_abort_loads.test)
This status variable indicates that an innodb buffer pool load never
completed and dumping at shutdown would result in an incomplete dump file.
This status variable is set to 1 once a buffer pool loads. Upon a successful
load this status variable returns to 0.
With this status variable set, the system variable
innodb_buffer_pool_dump_at_shutdown==1 will have no effect as dumping after
an incomplete load will generate a less complete dump file than the current
one.
If a user aborts a buffer pool load by changing the system variable
innodb_buffer_pool_load_abort=1 will cause the the status variable
innodb_buffer_pool_load_incomplete to remain set to 1.
A shutdown that occurs while innodb is loading the buffer pool will
not save the buffer pool on shutdown.
A user may indirectly set innodb_buffer_pool_load_incomplete
to 0 by:
* Forcing a load, by setting innodb_buffer_pool_load_now=ON, or
* Forcing a dump, by setting innodb_buffer_pool_dump_now=ON
This will enable the next dump on shutdown to complete.
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
Before MDEV-12288 in MariaDB 10.3.1, InnoDB used to partition
the persistent transaction undo log into insert_undo and update_undo.
MDEV-12288 repurposes the update_undo as the single undo log.
In order to support an upgrade from earlier MariaDB versions,
the insert_undo is recovered in data structures, called old_insert.
An assertion failure occurred in TrxUndoRsegsIterator::set_next()
when an incomplete transaction was recovered with both insert_undo
and update_undo log. This could be easily demonstrated by starting
./mysql-test-run --manual-gdb innodb.read_only_recovery
in MariaDB 10.2, and after the first kill, start up the MariaDB 10.3
server with the same parameters.
The problem is that MariaDB 10.3 would roll back the recovered
transaction, and finally "commit" it twice (with all changes to
data rolled back), both insert_undo and update_undo with the same
commit end identifier (trx->no).
Our fix is to introduce a "commit number" that comprises two components:
(trx->no << 1 | !old_insert). In this way, the assertion in the purge
subsystem can be relaxed so that only the trx->no component must match.
TrxUndoRsegs::append(): Remove.
TrxUndoRsegsIterator::set_next(): Add a debug assertion that
demonstrates that the merging of rollback segments never occurs.
Since MDEV-12289 or earlier, MariaDB 10.2 will not make any
temporary undo log accessible to the purge subsystem.
(Also MySQL 5.7 would skip the purge of any undo log for
temporary tables, but not before parsing and buffering those
temporary undo log records.)
Also, remove the field undo_rseg_space.
Apparently its purpose was to avoid problems with
temporary undo logs, which MySQL 5.7 unnecessarily adds to
the purge system. (Temporary undo log records are not purged.)
MariaDB 10.2 fixed this in MDEV-12289 or earlier.
purge_iter_t::operator<=(): Ordering comparison.
This replaces trx_purge_check_limit() with the difference that
we are not comparing undo_rseg_space. (In MariaDB, temporary
undo logs do not enter the purge subsystem at all.)
purge_sys_t::done: Remove. This was not used for anything.
purge_sys_t::tail: Renamed from purge_sys_t::iter.
purge_sys_t::head: Renamed from purge_sys_t::limit.
Analysis:- The problem is the change in the implementation of wait_for_listen
in wsrep_sst_xtrabackup-v2.sh. The new script uses lsof which will always
exit with an error code if it can't find all the items, and because the
script has the -e option set in the hashbang line (#!/bin/bash -ue), the
script will abort right after running lsof if lsof can't find even a single
item among all the items listed in its arguments. This will happen even if
socat is running and listening, because it can't find nc. The loop in
wait_for_listen will therefore always quit after one iteration without
writing the "ready" line to signal the parent.
Solution:- We will or the lsof with true.
Patch Credit :Daniel Black and David Wang
The change N7 in MDEV-15340 (see the commit message) introduced
a regression in how CAST(AS TIME), HOUR(), TIME_TO_SEC() treat datetimes
'0000-00-DD mm:hh:ss' (i.e. with zero YYYYMM part and a non-zero day).
These functions historically do not mix days to hours on datetime-to-time
conversion. Implementations of the underlying methods used get_arg0_time()
to fetch MYSQL_TIME. After MDEV-15340, get_arg0_time() went through the
Time() constructor, which always adds '0000-00-DD' to hours automatically
(as in all other places in the code we do mix days to hours).
Changes:
1. Extending Time() to make it possible to choose a desired way of treating
'0000-00-DD' (ignore or mix to hours) on datetime-to-time conversion.
Adding a helper class Time::Options for this, which now describes two aspects
of Time() creation:
1. Flags for get_date()
2. Days/hours mixing behavior.
2. Removing Item_func::get_arg0_time(). Using Time() directly
in all affected classes. Forcing Time() to ignore (rather than mix)
'0000-00-DD' in these affected classes by passing a suitable Options value.
3. Adding Time::to_seconds(), to reuse the code between
Item_func_time_to_sec::decimal_op() and Item_func_time_to_sec::int_op().
4. Item_func::get_arg0_date() now returns only a datetime value,
with automatic time-to-datetime conversion if needed. An assert was
added to catch attempts to pass TIME_TIME_ONLY to get_arg0_date().
All callers were checked not to pass TIME_TIME_ONLY, this revealed
a bug MDEV-15363.
5. Changing Item_func_last_day::get_date() to remove the TIME_TIME_ONLY flag
before calling get_arg0_date(). This fixes MDEV-15363.
a) We do not need this on Windows, and it is not clear what it does,inside
service.
b) It hinders debugging.
mysql-test-run.pl --debugger=vsjitdebugger more often than , would stop here
throwing "invalid handle" exception.
elements of BIGINT or YEAR type in the IN list reaches in_predicate_conversion_threshold
The bug appears at the prepare stage when IN-predicate with the long list
of values is converted into IN-subquery. It happens because values in the
right operand of the IN-predicate that have BIGINT or YEAR types are converted
into the Item_int_with_ref.
To fix it in the procedure Item_func_in::create_value_list_for_tvc
real_item() is taken for each value in the right operand of the IN-predicate.
This is based on a prototype by
Thirunarayanan Balathandayuthapani <thiru@mariadb.com>.
Binlog and Galera write-set replication information was written into
TRX_SYS page on each commit. Instead of writing to the TRX_SYS during
normal operation, InnoDB can make use of rollback segment header pages,
which are already being written to during a commit.
The following list of fields in rollback segment header page are added:
TRX_RSEG_BINLOG_OFFSET
TRX_RSEG_BINLOG_NAME (NUL-terminated; empty name = not present)
TRX_RSEG_WSREP_XID_FORMAT (0=not present; 1=present)
TRX_RSEG_WSREP_XID_GTRID
TRX_RSEG_WSREP_XID_BQUAL
TRX_RSEG_WSREP_XID_DATA
trx_sys_t: Introduce the fields
recovered_binlog_filename, recovered_binlog_offset, recovered_wsrep_xid.
To facilitate upgrade from older mysql or mariaDB versions, we will read
the information in TRX_SYS page. It will be overridden by the
information that we find in rollback segment header pages.
Mariabackup --prepare will read the metadata from the rollback
segment header pages via trx_rseg_array_init(). It will still
not read any undo log pages or recover any transactions.
trx_sys_t::rseg_history_len: Make private, and clarify the
documentation.
trx_sys_t::history_size(): Read rseg_history_len.
trx_sys_t::history_insert(), trx_sys_t::history_remove(),
trx_sys_t::history_add(): Update rseg_history_len.
the non-recursive CTE via prepared statement
The problem appears as the column names of the CTE were allocated on the
wrong MEMROOT and after the preparation of the statement they disappear.
To fix it in the procedure With_element::rename_columns_of_derived_unit
the CTE column names are now allocated in the permanent MEMROOT for the
prepared statements and stored procedures.
The problem was that Item_func_hybrid_field_type::get_date() did not
convert the result to the correct data type, so MYSQL_TIME::time_type
of the get_date() result could be not in sync with field_type().
Changes:
1. Adding two new classes Datetime and Date to store MYSQL_TIMESTAMP_DATETIME
and MYSQL_TIMESTAMP_DATE values respectively
(in addition to earlier added class Time, for MYSQL_TIMESTAMP_TIME values).
2. Adding Item_func_hybrid_field_type::time_op().
It performs the operation using TIME representation,
and always returns a MYSQL_TIME value with time_type=MYSQL_TIMESTAMP_TIME.
Implementing time_op() for all affected children classes.
3. Fixing all implementations of date_op() to perform the operation
using strictly DATETIME representation. Now they always return a MYSQL_TIME
value with time_type=MYSQL_TIMESTAMP_{DATE|DATETIME},
according to the result data type.
4. Removing assignment of ltime.time_type to mysql_timestamp_type()
from all val_xxx_from_date_op(), because now date_op() makes sure
to return a proper MYSQL_TIME value with a good time_type (and other member)
5. Adding Item_func_hybrid_field_type::val_xxx_from_time_op().
6. Overriding Type_handler_time_common::Item_func_hybrid_field_type_val_xxx()
to call val_xxx_from_time_op() instead of val_xxx_from_date_op().
7. Modified Item_func::get_arg0_date() to return strictly a TIME value
if TIME_TIME_ONLY is passed, or return strictly a DATETIME value otherwise.
If args[0] returned a value of a different temporal type,
(for example a TIME value when TIME_TIME_ONLY was not passed,
or a DATETIME value when TIME_TIME_ONLY was passed), the conversion
is automatically applied.
Earlier, get_arg0_date() did not guarantee a result in
accordance to TIME_TIME_ONLY flag.
