PROBLEM
By design stats estimation always reading uncommitted data. In this scenario
an uncommitted transaction has deleted all rows in the table. In Innodb
uncommitted delete records are marked as delete but not actually removed
from Btree until the transaction has committed or a read view for the rows
is present.While calculating persistent stats we were ignoring the delete
marked records,since all the records are delete marked we were estimating
the number of rows present in the table as zero which leads to bad plans
in other transaction operating on the table.
Fix
Introduced a system variable called innodb_stats_include_delete_marked
which when enabled includes delete marked records for stat
calculations .
Apparently, WL#8149 QA did not cover the code changes made to
online table rebuild (which was introduced in MySQL 5.6.8 by WL#6255)
for ROW_FORMAT=REDUNDANT tables.
row_log_table_low_redundant(): Log the new values of indexed virtual
columns (ventry) only once.
row_log_table_low(): Assert that if o_ventry is specified, the
logged operation must not be ROW_T_INSERT, and ventry must be specified
as well.
row_log_table_low(): When computing the size of old_pk, pass v_entry=NULL to
rec_get_converted_size_temp(), to be consistent with the subsequent call
to rec_convert_dtuple_to_temp() for logging old_pk. Assert that
old_pk never contains information on virtual columns, thus proving that this
change is a no-op.
RB: 13822
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
Problem:
=======
Autoincrement value gives duplicate values because of the following reasons.
(1) In InnoDB handler function, current autoincrement value is not changed
based on newly set auto_increment_increment or auto_increment_offset variable.
(2) Handler function does the rounding logic and changes the current
autoincrement value and InnoDB doesn't aware of the change in current
autoincrement value.
Solution:
========
Fix the problem(1), InnoDB always respect the auto_increment_increment
and auto_increment_offset value in case of current autoincrement value.
By fixing the problem (2), handler layer won't change any current
autoincrement value.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 13748
When a table has no PRIMARY KEY, but there is a UNIQUE INDEX
defined on NOT NULL columns that are not column prefixes,
that unique index must be treated as the primary key.
This property was being violated by InnoDB when a column was changed
to NOT NULL, such that a UNIQUE INDEX on that column became eligible
to being treated as a primary key.
innobase_create_key_defs(): Instead of checking each ADD [UNIQUE] INDEX
request, check if a GEN_CLUST_INDEX can be replaced with any unique index
in the altered_table definition. So, we can have new_primary even
if n_add==0.
prepare_inplace_alter_table_dict(): When the table is not being rebuilt,
assert that TABLE_SHARE::primary_key is not changing.
RB: 13595
Reviewed-by: Kevin Lewis <kevin.lewis@oracle.com>
Problem:
=======
High priority transaction can't able to kill the blocking transaction
when foreign keys are involved. trx_kill_blocking() missing while checking
the foreign key constraint.
Fix:
===
Add trx_kill_blocking() while checking for the foreign key constraint.
Reviewed-by: Debarun Banerjee <debarun.banerjee@oracle.com>
RB: 13579
Analysis: In row_log_table_delete(), extern size could be greater
than 2 bytes int if there are enough index on blob columns.
Solution: Use 4 bytes int other than 2 bytes for extern size.
Reviewed-by: Marko Mäkelä <marko.makela@oracle.com>
RB: 13573
BUG#23742339 FAILING ASSERTION: SYM_NODE->TABLE != NULL
Analysis: When we access fts aux tables in information schema,the
fts aux tables are dropped by DROP DATABASE in another session.
Solution: Drop parent table if it's a fts aux table, and drop
table will drop fts aux tables together.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 13264
Analysis:
the old table is dropped, just after it's added into drop list,
and new table with the same name is created, then we try to drop
the new table in background.
Solution:
Don't drop a table in background if table->to_be_dropped is false.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 13414
Analysis:
When we access fts_internal_tbl_name in i_s_fts_config_fill (),
it can be set to NULL by another session.
Solution:
Define fts_internal_tbl_name2 for global variable innodb_ft_aux_table,
if it's NULL, set fts_internal_tbl_name to "default".
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
RB: 13401
Problem:
=======
Inplace alter algorithm determines the table to be rebuild if the table
undergoes row format change, key block size if handler flag contains only
change table create option. If alter with inplace ignore flag operations and change table create options then it leads to table rebuild operation.
Solution:
========
During the check for rebuild, ignore the inplace ignore flag and check for
table create options.
Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>
Reviewed-by: Marko Makela <marko.makela@oracle.com>
RB: 13172
This patch fixed some problems that occurred with subqueries that
contained directly or indirectly recursive references to recursive CTEs.
1. A [NOT] IN predicate with a constant left operand and a non-correlated
subquery as the right operand used in the specification of a recursive CTE
was considered as a constant predicate and was evaluated only once.
Now such a predicate is re-evaluated after every iteration of the process
that produces the records of the recursive CTE.
2. The Exists-To-IN transformation could be applied to [NOT] IN predicates
with recursive references. This opened a possibility of materialization
for the subqueries used as right operands. Yet, materialization
is prohibited for the subqueries if they contain a recursive reference.
Now the Exists-To-IN transformation cannot be applied for subquery
predicates with recursive references.
The function st_select_lex::check_subqueries_with_recursive_references()
is called now only for the first execution of the SELECT.
buf_flush_write_block_low(): Acquire the tablespace reference once,
and pass it to lower-level functions. This is only a start; further
calls may be removed.
fil_decompress_page(): Remove unsafe use of fil_space_get_by_id().
fil_crypt_thread(): Do invoke fil_crypt_complete_rotate_space()
when the tablespace is about to be dropped. Also, remove a redundant
check whether rotate_thread_t::space is NULL. It can only become
NULL when fil_crypt_find_space_to_rotate() returns false, and in
that case we would already have terminated the loop.
fil_crypt_find_page_to_rotate(): Remove a redundant check for
space->crypt_data == NULL. Once encryption metadata has been
created for a tablespace, it cannot be removed without dropping
the entire tablespace.
Alias the InnoDB ulint and lint data types to size_t and ssize_t,
which are the standard names for the machine-word-width data types.
Correspondingly, define ULINTPF as "%zu" and introduce ULINTPFx as "%zx".
In this way, better compiler warnings for type mismatch are possible.
Furthermore, use PRIu64 for that 64-bit format, and define
the feature macro __STDC_FORMAT_MACROS to enable it on Red Hat systems.
Fix some errors in error messages, and replace some error messages
with assertions.
Most notably, an IMPORT TABLESPACE error message in InnoDB was
displaying the number of columns instead of the mismatching flags.
Define UNIV_WORD_SIZE as a simple alias to SIZEOF_SIZE_T.
In MariaDB 10.0 and 10.1, it was incorrectly defined as 4 on
64-bit Windows.
MONITOR_OS_PENDING_READS, MONITOR_OS_PENDING_WRITES: Enable by default.
os_n_pending_reads, os_n_pending_writes: Remove.
Use the monitor counters instead.
Use older version of the SDK generally, because the newer ones break
with older cmake.
On Macs, use newer version, to fix mac specific the build error.
Update AWS SDK version from 1.0.8 to 1.0.100
Commit b64910ce27 (MDEV-12453)
enabled AWS_SDK to build correctly on buildbot.
Travis still had build faults like below despite many common elements
between the builds;
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:79:9: error: no member named 'strcoll' in the global namespace; did you mean 'strtoll'?
[ 24%] Building CXX object storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/db/internal_stats.cc.o
using ::strcoll;
~~^
/usr/include/stdlib.h:169:3: note: 'strtoll' declared here
strtoll(const char *__str, char **__endptr, int __base);
^
When MDEV-6076 repurposed the field PAGE_MAX_TRX_ID, it was assumed
that the field always was 0 in the clustered index of old data files.
This was not the case in IMPORT TABLESPACE (introduced in MySQL 5.6
and MariaDB 10.0), which is writing the transaction ID to all index
pages, including clustered index pages.
This means that on a data file that was at some point of its life
IMPORTed to an InnoDB instance, MariaDB 10.2.4 or later could interpret
the transaction ID as a persistent AUTO_INCREMENT value.
This also means that future changes that repurpose PAGE_MAX_TRX_ID
in the clustered index may cause trouble with files that were imported
at some point of their life.
There is a separate minor issue that InnoDB is writing PAGE_MAX_TRX_ID
to every secondary index page, even though it is only needed on leaf
pages. From now on we will write PAGE_MAX_TRX_ID as 0 to non-leaf pages,
just to be able to keep stricter debug assertions.
btr_root_raise_and_insert(): Reset the PAGE_MAX_TRX_ID field on non-root
pages of the clustered index, and on the no-longer-leaf root page of
secondary indexes.
AbstractCallback::is_root_page(): Remove. Use page_is_root() instead.
PageConverter::update_index_page(): Reset the PAGE_MAX_TRX_ID to 0
on other pages than the clustered index root page or secondary index
leaf pages.
Fixed handling of default values with cached temporal functions so that the
CREATE TABLE statement now succeeds.
Fixed virtual column session cleanup.
Fixed the error message.
Added quoting of date/time values in cases when this was omitted.
Added a test case in default.test.
Updated test result files.
Fixed the bug by failing the statement with an error message that explains
that an auto-increment column may not be used in an expression for a
check constraint.
Added a test case in check_constraint.test.
Updated existing tests and results.
PARS_INTEGER_TOKEN: Remove. The lexer returns only PARS_INT_TOKEN.
PARS_FIXBINARY_LIT, PARS_BLOB_LIT: Remove. These are never returned
by the lexer. In sym_tab_add_bound_lit(), use PARS_STR_LIT.
dict_index_is_sec_or_ibuf(): Use a single arithmetic expression.
rtr_split_page_move_rec_list(): Remove a redundant condition on
dict_index_is_sec_or_ibuf(). This function is always invoked on
a spatial index, which also is a secondary index.
Remove clang-3.8 which doesn't have a repository on apt.llvm.org any
more.
For OSX, xcode8.3 is explicitly specified.
/usr/local/Cellar is used as a cache repository to save brew install
time on OSX (and /usr/local was too big).
Debian autobake.sh is moved to a matrix include.
Other branches of the matrix build test other test suites.
An Ubuntu galera is downloaded and used in the test suite.
TYPE=RelWithDebInfo used with the test to provide backtraces with line
numbers when crashes occur.
Build of PLUGIN_AWS_KEY_MANAGEMENT enabled in build.
Code supporting TYPE=Debug and -DWITH_ASAN=ON included by not enabled
due to large numbers of errors.
Running more tests in parallel (6) as container based builds seem to
support them. The test case timeout has been set to 2 minutes as large
stalls will put test cases over 50 minute interval.
ccache enabled where possible. Linux clang builds don't use them as the
minimum CMake version isn't there.
In case of error on opening VIEW (absent table for example) it is still possible to print its definition but some variable is not set (table_list->derived->derived) so it is better do not try to test it when there is safer alternative (table_list itself).
because on Windows it cannot properly append to the file,
doesn't use CreateFile(..., FILE_APPEND_DATA, ...)
this fixes main.shutdown failures on Windows
Annotate_rows_log_event again. When a new annotate event comes,
the server applies it first (which backs up thd->query_string),
then frees the old annotate event, if any. Normally there isn't.
But with sub-statements (e.g. triggers) new annotate event comes
before the first one is freed, so the second event backs up
thd->query_string that was set by the first annotate event. Then
the first event is freed, together with its query string. And then
the second event restores thd->query_string to this freed memory.
Fix: free old annotate event before applying the new one.
automatic shortening of a too-long non-unique key should
be not a warning, but a note. It's a normal optimization,
doesn't affect correctness, and should never be converted to
an error, no matter how strict sql_mode is.
in MYSQL_ADD_PLUGIN, do not add TARGET_LINK_LIBRARIES twice for the LINK_LIBRARIES parameter
It is usually harmless to add libraries twice.
However, aws_key_management uses -Wl,-whole-archive to workaround linker issues on Linux
If libraries are added twice with whole-archive, linking will fail complaining about duplicate symbols
use CMAKE_CXX_STANDARD to set C++11 flags with CMake 3.1+ (apples flags are somehow different from standard clang)
port htonbe16/32/64 macros for rocksdb
use reinterpret_cast<size_t> to cast macOS's pthread_t (pointer type) to size_t , for rocksdb
When a CTE referring to another CTE from the same with clause
was used twice then the server could not find the second CTE and
reported a bogus error message.
This happened because for any unit that was created as a clone of
a CTE specification the pointer to the WITH clause that owned this CTE
was not set.
