row_ins_clust_index_entry_low(): Invoke btr_set_instant() in the same
mini-transaction that has successfully inserted the metadata record.
In this way, if inserting the metadata record fails before any
undo log record was written for it, the index root page will remain
consistent.
innobase_instant_try(): Remove the btr_set_instant() call.
Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Matthias Leich
ha_innobase::check_if_supported_inplace_alter(): On ALTER_OPTIONS,
if innodb_file_per_table=1 and the table resides in the system tablespace,
require that the table be rebuilt (and moved to an .ibd file).
Reviewed by: Thirunarayanan Balathandayuthapani
Tested by: Matthias Leich
Remove ORACLE from the (session) sql_mode in connections made with sql
service to run init queries
The connection is new and the global variable value takes effect
rather than the session value from the caller of spider_db_init.
This should fix certain CI builds where the spider suite test files
and the main suite test files do not follow the same relative paths
relations as the mariadb source.
$MYSQLD_CMD uses .1 as the defaults-group-suffix, which could cause
the use of the default port (3306) or socket, which will fail in
environment where these defaults are already in use by another server.
Adding an extra --defaults-group-suffix=.1.1 does not help, because
the first flag wins.
So we use $MYSQLD_LAST_CMD instead, which uses the correct suffix.
The extra innodb buffer pool warning is irrelevant to the goal of the
test (running --wsrep-recover with --plug-load-add=ha_spider should
not cause hang)
Fix spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218) while
preventing regression on old ones (MDEV-30370, MDEV-29904)
Two things are changed:
First, Spider initialisation is made fully synchronous, i.e. it no
longer happens in a background thread. Adapted from the original fix
by nayuta for MDEV-27233. This change itself would cause failure when
spider is initialised early, by plugin-load-add, due to dependency on
Aria and udf function creation, which are fixed in the second and
third parts below. Requires SQL Service, thus porting earlier versions
requires MDEV-27595
Second, if spider is initialised before udf_init(), create udf by
inserting into `mysql.func`, otherwise do it by `CREATE FUNCTION` as
usual. This change may be generalised in MDEV-31401.
Also factor out some clean-up queries from deinit_spider.inc for use
of spider init tests.
A minor caveat is that early spider initialisation will fail if the
server is bootstrapped for the first time, due to missing `mysql`
database which needs to be created by the bootstrap script.
Removing procedures that were created and dropped during init.
This also fixes a race condition where mtr test with
plugin-load-add=ha_spider.so causes post test check to fail as it
expects the procedures to still be there.
There are several plugins in ha_spider: spider, spider_alloc_mem,
spider_wrapper_protocols, spider_rewrite etc.
INSTALL PLUGIN foo SONAME ha_spider causes all the other ones to be
installed by the init queries where foo is any of the plugins.
This introduces unnecessary complexiy. For example it reads
mysql.plugins to find all other plugins, causing the hack of moving
spider plugin init to a separate thread.
To install all spider related plugins, install soname ha_spider should
be used instead.
This also fixes spurious rows in mysql.plugin when installing say only
the spider plugin with `plugin-load-add=SPIDER=ha_spider.so`:
select * from mysql.plugin;
name dl
spider_alloc_mem ha_spider.so # should not be here
spider_wrapper_protocols ha_spider.so # should not be here
Adapted from part of the reverted commit
c160a115b8.
Problem:
========
- InnoDB should have two keys on the same column for the self
referencing foreign key relation.
Solution:
=========
- Allow self referential foreign key relation to work with one
key.
row_upd_clust_rec_by_insert(): If we are resuming from a lock wait,
reset the 'disowned' flag of the BLOB pointers in 'entry' that we
copied from 'rec' on which we had invoked btr_cur_disown_inherited_fields()
before the lock wait started. In this way, the inserted record with
the updated PRIMARY KEY value will have the BLOB ownership associated
with itself, like it is supposed to be.
Note: If the lock wait had been aborted, then rollback would have
invoked btr_cur_unmark_extern_fields() and no corruption would be possible.
Reviewed by: Vladislav Lesin
Tested by: Matthias Leich
trx_t::commit_in_memory(): Empty the detailed_error string, so that
FOREIGN KEY error messages from an earlier transaction will not be
wrongly reused in ha_innobase::get_error_message().
Reviewed by: Thirunarayanan Balathandayuthapani
- Commit fc31e3114b2538d152194d17ff0f0439db565634(MDEV-8179) doesn't
report the progress of inplace alter completely. It just does only
in row_merge_sort(). Removing the progress report function completely
This avoids the scenario in MDEV-32849, when the unlock happens after
the connection has been freed, say in rollback. This is done in 10.5+
after the commit a26700cca5.
It may or may not prevent potential other scenarios where spider has
locked something, then for some reason the statement needs to be
rolled back and spider frees the connection, and then spider proceeds
to use the freed connection. But at least we fix the regression
introduced by MDEV-30014 to 10.4 and bring 10.4 closer in parity with
10.5+.
mysql_prepare_alter_table(): Alter table should check whether
foreign key exists when it expected to exists and
report the error in early stage
dict_foreign_parse_drop_constraints(): Don't throw error if the
foreign key constraints doesn't exist when if exists is given
in the statement.
We remove the call to update spider persistent table stats (sts/crd)
in spider_free_share(). This prevents spider from opening and closing
further tables during close(), which fixes the following issues:
MDEV-28739: ha_spider::close() is called during tdc_start_shutdown(),
which is called after query_cache_destroy(). Closing the sts/crd Aria
tables will trigger a call to Query_cache::invalidate_table(), which
will attempt to use the query cache mutex structure_guard_mutex
destroyed previously.
MDEV-29421: during ha_spider::close(), spider_free_share() could
trigger another spider_free_share() through updating sts/crd table,
because open_table() calls tc_add_table(), which could trigger another
ha_spider::close()...
Since spider sts/crd system tables are only updated here, there's no
use for these tables any more, and we remove all uses of these tables
too.
The removal should not cause any performance issue, as in memory
spider table stats are only updated based on a time
interval (spider_sts_interval and spider_crd_interval), which defaults
to 10 seconds. It should not affect accuracy either, due to the
infrequency of server restart. And inaccurate stats are not a problem
for optimizer anyway.
To be on the safe side, we defer the removal of the spider sts/crd
tables themselves to future.
btr_cur_update_in_place(): Update the DB_TRX_ID,DB_ROLL_PTR also on the
compressed copy of the page. In a test case, a server built with
cmake -DWITH_INNODB_EXTRA_DEBUG=ON would crash in page_zip_validate()
due to the inconsistency. In a normal debug build, a different assertion
would fail, depending on when the uncompressed page was restored from
the compressed page.
In MariaDB Server 10.5, this bug had already been fixed by
commit b3d02a1fcf (MDEV-12353).
Problem:
========
- InnoDB fails to free the allocated buffer of stored cursor
when information schema query is interrupted.
Solution:
=========
- In case of error handling, information schema query should free
the allocated buffer to store the cursor.
This will avoid issues like MDEV-32486
IDs used in
- spider_alloc_calc_mem_init()
- spider_string::init_calc_mem()
- spider_malloc()
- spider_bulk_alloc_mem()
- spider_bulk_malloc()
This fixes MDEV-30014, MDEV-29456, MDEV-29667, and MDEV-30049.
The server may ask storage engines to unlock when the original sql
command is not UNLOCK. This patch makes sure that spider honours these
requests, so that the server has the correct idea which tables are
locked and which are not.
MDEV-29456, MDEV-29667, MDEV-30049: a later LOCK statement would, as
the first step, unlock locked tables and clear the OPTION_TABLE_LOCK
bit in thd->variables.option_bits, as well as locked_tables_list,
indicating no tables are locked. If Spider does not unlock because the
sql command is not UNLOCK, and if after this the LOCK statement fails
to lock any tables, these indications that no tables are locked
remains, so a later UNLOCK TABLES; statement would not try to unlock
any table. Causing later statements requiring mdl locks to hang on
waiting until lock_wait_timeout (default 1h) has passed.
MDEV-30014: when a LOCK statement tries to lock more than one tables,
say t2 and t3 as in mdev_30014.test, and t2 succeeds but t3 fails, the
sql layer would try to undo by unlocking t2, and again, if Spider does
not honour this request, the sql layer would assume t2 has been
unlocked, but later actions on t2 or t2's remote table could hang on
waiting for the mdl.
Spider populates its lock lists (a hash) in store_lock(), and normally
clears them in the actual lock_tables(). However, if lock_tables()
fails, there's no reset_lock() method for storage engine handlers,
which can cause bad things to happen. For example, if one of the table
involved is dropped and recreated, or simply TRUNCATEd, when executing
LOCK TABLES again, the lock lists would be queried again in
store_lock(), which could cause access to freed space associated with
the dropped table.
Spider GBH's query rewrite of table joins is overly complex and
error-prone. We replace it with something closer to what
dbug_print() (more specifically, print_join()) does, but catered to
spider.
More specifically, we replace the body of
spider_db_mbase_util::append_from_and_tables() with a call to
spider_db_mbase_util::append_join(), and remove downstream append_X
functions.
We make it handle const tables by rewriting them as (select 1). This
fixes the main issue in MDEV-26247.
We also ban semijoin from spider gbh, which fixes MDEV-31645 and
MDEV-30392, as semi-join is an "internal" join, and "semi join" does
not parse, and it is different from "join" in that it deduplicates the
right hand side
Not all queries passed to a group by handler are valid (MDEV-32273),
for example, a join on expr may refer outer fields not in the current
context. We detect this during the handler creation when walking the
join. See also gbh_outer_fields_in_join.test.
It also skips eliminated tables, which fixes MDEV-26193.
Spider gbh query rewrite should get table for fields in a simple way.
Add a method spider_fields::find_table that searches its table holders
to find table for a given field. This way we will be able to get rid
of the first pass during the gbh creation where field_chains and
field_holders are created.
We also check that the field belongs to a spider table while walking
through the query, so we could remove
all_query_fields_are_query_table_members(). However, this requires an
earlier creation of the table_holder so that tables are added before
checking. We do that, and in doing so, also decouple table_holder and
spider_fields
Remove unused methods and fields. Add comments.
Two methods from spider_fields. There are probably more of these
conn_holder related methods that can be removed
reappend_tables_part()
reappend_tables()
When computing vcol expression some items use current_thd and that was
not set in MyISAM repair thread. Since all the repair threads belong
to one connection and items should not write into THD we can utilize
table THD for that.
Index values for row_start/row_end was wrongly calculated for inplace
ALTER for some layout of virtual fields.
Possible impact
1. history row is not detected upon build clustered index for
inplace ALTER which may lead to duplicate key errors on
auto-increment and FTS index add.
2. foreign key constraint may falsely fail.
3. after inplace ALTER before server restart trx-based system
versioning can cause server crash or incorrect data written upon
UPDATE.
Before MariaDB 10.3.5, the binlog position was stored in the TRX_SYS page,
while after it is stored in rollback segments. There is code to read the
legacy position from TRX_SYS to handle upgrades. The problem was if the
legacy position happens to compare larger than the position found in
rollback segments; in this case, the old TRX_SYS position would incorrectly
be preferred over the newer position from rollback segments.
Fixed by always preferring a position from rollback segments over a legacy
position.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
This commit can cause the wrong (old) binlog position to be recovered by
mariabackup --prepare. It implements that the value of the FIL_PAGE_LSN is
compared to determine which binlog position is the last one and should be
recoved. However, it is not guaranteed that the FIL_PAGE_LSN order matches the
commit order, as is assumed by the code. This is because the page LSN could be
modified by an unrelated update of the page after the commit.
In one example, the recovery first encountered this in trx_rseg_mem_restore():
lsn=27282754 binlog position (./master-bin.000001, 472908)
and then later:
lsn=27282699 binlog position (./master-bin.000001, 477164)
The last one 477164 is the correct position. However, because the LSN
encountered for the first one is higher, that position is recovered instead.
This results in too old binlog position, and a newly provisioned slave will
start replicating too early and get duplicate key error or similar.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
In commit 75e82f71f1 the code to
rename internal tables for FULLTEXT INDEX that had been
created on Microsoft Windows using incompatible names
was removed. Let us also remove the related fault injection.
Problem:
=======
- InnoDB fails to find the foreign key index for the newly
added foreign key relation. This is caused by commit
5f09b53bdb (MDEV-31086).
FIX:
===
In check_col_is_in_fk_indexes(), while iterating through
the newly added foreign key relationship, InnoDB should
consider that foreign key relation may not have foreign index
when foreign key check is disabled.
When mysql/mysql-server@0c954c2289
added a plugin interface for FULLTEXT INDEX tokenization to MySQL 5.7,
fts_tokenize_ctx::processed_len got a second meaning, which is only
partly implemented in row_merge_fts_doc_tokenize().
This inconsistency could cause a crash when using FULLTEXT...WITH PARSER.
A test case that would crash MySQL 8.0 when using an n-gram parser and
single-character words would fail to crash in MySQL 5.7, because the
buf_full condition in row_merge_fts_doc_tokenize() was not met.
This change is inspired by
mysql/mysql-server@38e9a0779a
that appeared in MySQL 5.7.44.
Also fixes: MDEV-30050 Inconsistent results of DISTINCT with NOPAD
Problem:
Key segments for CHAR columns where compared using strnncollsp()
for engines MyISAM and Aria.
This did not work correct in case if the engine applyied trailing
space compression.
Fix:
Replacing ha_compare_text() calls to new functions:
- ha_compare_char_varying()
- ha_compare_char_fixed()
- ha_compare_word()
- ha_compare_word_prefix()
- ha_compare_word_or_prefix()
The code branch corresponding to comparison of CHAR column keys
(HA_KEYTYPE_TEXT segment type) now uses ha_compare_char_fixed()
which calls strnncollsp_nchars().
This patch does not change the behavior for the rest of the code:
- comparison of VARCHAR/TEXT column keys
(HA_KEYTYPE_VARTEXT1, HA_KEYTYPE_VARTEXT2 segments types)
- comparison in the fulltext code
Problem:
========
InnoDB fails to find the foreign key index for the
foreign key relation in the table while iterating the
foreign key constraints during alter operation. This is
caused by commit 5f09b53bdb
(MDEV-31086).
Fix:
====
In check_col_is_in_fk_indexes(), while iterating through
the foreign key relationship, InnoDB should consider that
foreign key relation may not have foreign index when
foreign key check is disabled.
fil_page_compress_low returns 0 for both innodb_compression_algorithm=0
and where there is compression errors. On the two callers to this
function, don't increment the compression errors if the algorithm was
none.
Reviewed by: Marko Mäkelä
While checking for altered column in foreign key constraints,
InnoDB fails to ignore virtual columns. This issue caused
by commit 5f09b53bdb4e973e7c7ec2c53a24c98321223f98(MDEV-31086).
