- Purge thread is trying to access the table name when other thread
does rename of the table name. It leads to heap use after free error
by purge thread. purge thread should check whether the table name
is temporary while holding dict_sys.mutex.
Creating a temporary table with Spider is non-sense because a Spider
table cannot hold any physical data and it requires an additional
effort to manage even if it is configured correctly.
Set HTON_TEMPORARY_NOT_SUPPORTED to spider_hton->flags.
Reviewed-by: nayuta.yanagisawa@hey.com
Co-authored-by: d8sk4ueun@gmail.com
dict_acquire_mdl_shared(): Invoke the correct variant
dict_table_t::parse_name<true>() also when trylock=true,
that is, we are being called from fts_optimize_sync_table().
Ever since commit 82b7c561b7 (MDEV-24258)
this code was prone to a hang. If another thread requested an
exclusive dict_sys.latch between the time
dict_acquire_mdl_shared<trylock=true>() acquired a shared dict_sys.latch
and dict_table_t::parse_name<false>() was trying to acquire another
shared dict_sys.latch, InnoDB would get into an infinite livelock
of threads, because the futex-based srw-lock implementation prioritizes
exclusive latch requests.
dict_table_t::parse_name(): Rename the template parameter dict_locked
into dict_frozen.
buf_page_t::set_state(): Relax a debug assertion. It is fine to update
a read-fixed block descriptor to be both read-fixed and buffer-fixed.
buf_pool_t::watch_unset(): Fix some incorrect logic that was implemented
in commit e9e6db9355.
Thanks to Elena Stepanova for the test case.
The check on the SQL command type, in ha_spider::external_lock() is deleted
by e954d9de. This resulted in the wrong call of spider_internal_start_trx()
(and thus Ha_trx_info::register_ha()).
I reverted the check and refactored ha_spider::external_lock().
The original query "SELECT IF(COUNT(a.`id`)>=0,'Y','N') FROM t" is
transformed to "SELECT COUNT(a.`id`), IF(ref >= 0, 'Y', 'N') FROM t",
where ref is Item_ref to "COUNT(a.`id`)", by split_sum_func().
Spider walks the item list twice, invoking spider_db_print_item_type().
The first invocation is in spider_create_group_by_handler() with
str == NULL. The second one is in spider_group_by_handler::init_scan()
with str != NULL.
spider_db_print_item_type() prints nothing at the first invocation,
and it prints item at the second invocation. However, at the second
invocation, the above mentioned ref to "COUNT(a.`id`)" points to
a field in a temporary table where the result will be stored. Thus,
to look behind the item_ref, Spider need to generate the query earlier.
A possible fix would be to generate a query to send in
spider_create_group_by_handler(). However, the fix requires a
considerable amount of changes of the Spider's GROUP BY handler.
I'd like to avoid that.
So, I fix the problem by not to use the GROUP BY handler when a
query contains Item_ref whose table_name, name, and alias_name_used
are not set.
This failure was caused by MDEV-25975, which removed the parameter
innodb_disallow_writes.
Added a check for wsrep_sst_disable_writes to the function
ibuf_merge_in_background().
- There is a race condition occurs between purge thread and DDL.
So purge thread can increment n_ref_count even after DDL does
purge_sys_t::stop_FTS().
- dict_table_open_on_id for purge thread should check
purge_sys.must_wait_FTS() before acquring the table.
- purge_sys.stop_FTS() does acquire dict_sys.latch for setting
the purge system flag and check table ref count on auxilary tables.
The Shared/Update/Exclusive locks that were introduced in
commit 03ca6495df (MDEV-24142)
did not work correctly when a futex-like system call interface
was not available.
On all tested implementations (IBM AIX as well as FreeBSD and GNU/Linux
with the futex interface artificially disabled), the old implementation
would cause hangs in some SPATIAL INDEX tests (innodb_gis suite).
On FreeBSD, a hang was also observed in an encryption test.
We will simply emulate the futex system calls with a single mutex
and two condition variables, one for each wait queue. The condition
variables basically shadow the lock words and are used as wait queues,
just like the futex system calls would be.
The storage overhead of ssux_lock_impl will be increased by 32 bits
when using SUX_LOCK_GENERIC. Compared to the futex-based implementation,
the SUX_LOCK_GENERIC implementation has an overhead of
sizeof(pthread_mutex_t)+2*sizeof(pthread_cond_t).
rw_lock: Remove all SUX_LOCK_GENERIC extensions.
pthread_mutex_wrapper: A simple wrapper of pthread_mutex that
implements srw_spin_mutex and srw_mutex for SUX_LOCK_GENERIC.
srw_mutex_impl: Define this also for SUX_LOCK_GENERIC, but in
that case add the fields mutex, cond.
ssux_lock_impl: Define for SUX_LOCK_GENERIC with a minimal difference:
adding readers_cond.
We will remove the parameter innodb_disallow_writes because it is badly
designed and implemented. The parameter was never allowed at startup.
It was only internally used by Galera snapshot transfer.
If a user executed
SET GLOBAL innodb_disallow_writes=ON;
the server could hang even on subsequent read operations.
During Galera snapshot transfer, we will block writes
to implement an rsync friendly snapshot, as follows:
sst_flush_tables() will acquire a global lock by executing
FLUSH TABLES WITH READ LOCK, which will block any writes
at the high level.
sst_disable_innodb_writes(), invoked via ha_disable_internal_writes(true),
will suspend or disable InnoDB background tasks or threads that could
initiate writes. As part of this, log_make_checkpoint() will be invoked
to ensure that anything in the InnoDB buf_pool.flush_list will be written
to the data files. This has the nice side effect that the Galera joiner
will avoid crash recovery.
The changes to sql/wsrep.cc and to the tests are based on a prototype
that was developed by Jan Lindström.
Reviewed by: Jan Lindström
- InnoDB bulk insert operation fails to rollback when it detect
DB_DUPLICATE_KEY error. It leads to orphaned records in primary
indexes. Consecutive update/delete operation assumes that record
should exist in secondary index and it leads to failure.
- After MDEV-24621, InnoDB does buffer the insert bulk operation
for all indexes expect spatial one. But it leads to search the
primary key lookup and it leads to failure. So InnoDB should avoid
bulk insert when table has spatial index involved.
The issue is caused by 59a0236da4 commit.
The initial intention of the commit was to speed up
"mariabackup --prepare".
The call stack of binlog position reading is the following:
▾ trx_rseg_mem_restore
▾ trx_rseg_array_init
▾ trx_lists_init_at_db_start
▸ srv_start
Both trx_lists_init_at_db_start() and trx_rseg_mem_restore() contain
special cases for srv_operation == SRV_OPERATION_RESTORE condition, and
on this condition only rseg headers are read to parse binlog position.
Performance impact is not so big.
The solution is to revert 59a0236da4.
ib_id_t is a uint64. On AIX this isn't a long long unsigned and to
prevent the compile warnings and potential wrong type, the UINT64PFx
defination is corrected.
As INT64PF is unused (last use, xtradb in 10.2), it is removed to
remove the confusion that INT64PF and UINT64PFx would be different
types otherwise.
CMAKE_SYSTEM_PROCESSOR on AIX is "powerpc". To
deconflict with the Linux 32bit arch of the same
name, CMAKE_SYSTEM_NAME was used in the CMakeLists.txt
test to enable -mhtm in the same way that was required
for Linux ppc64{,le} compilers in MDEV-27936
Configuring UDFs via plugin variables looks not a good idea.
The more variables Spider has, the more complex it becomes.
Further, I expect that only a few users use Spider UDFs.
Deprecate the following plugin variables regarding Spider UDFs:
* spider_udf_ds_bulk_insert_rows
* spider_udf_ds_table_loop_mode
* spider_udf_ds_use_real_table
* spider_udf_ct_bulk_insert_interval
* spider_udf_ct_bulk_insert_rows
spider_udf_table_lock_mutex_count and spider_udf_table_mon_mutex_count
are also for tweaking UDFs but they are already read-only. So,
there is no need to deprecate them.
"#ifdef WITH_PARTITION_STORAGE_ENGINE ... #endif" appears frequently
in the Spider code base. However, there is no need to maintain such
ifdefs because Spider is disabled if the partitioning engine is disabled.
This was noticed as part of verifying
MDEV-28186 "crash on startup after crash while regular use"
but is probably not related to the users issue.
Still good to have it fixed
mariabackup does not load dictionary during backup, but it loads
tablespaces, that is why fil_system.max_assigned_id is not set with
dict_check_tablespaces_and_store_max_id(). There is no sense to issue the
warning during backup.
The comparison on the checkpoint age (number of log bytes
written since the previous checkpoint) is inaccurate, because
the previous FILE_CHECKPOINT record could span two 512-byte
log blocks, which will cause the LSN to increase by the size of the
log block header and footer.
We will still generate a redudant checkpoint if the previous
checkpoint wrote some FILE_MODIFY records before the FILE_CHECKPOINT
record.
Whenever we retrieve an older version for READ COMMITTED,
it is better to release the undo page latches
so that we can freely move to the next clustered index record
without potentially violating any latching order.
Only one checkpoint may be in progress at a time.
The counter log_sys.n_pending_checkpoint_writes
was being protected by log_sys.mutex.
Let us replace it with the Boolean log_sys.checkpoint_pending.
srv_start(): Set srv_startup_is_before_trx_rollback_phase before
starting the buf_flush_page_cleaner() thread, so that it will not
invoke log_checkpoint() before the log file has been created.
This race condition was reproduced with https://rr-project.org.
This fixes up commit 15efb7ed48
buf_pool_t::watch_unset(): Reorder some code so that
no warning will be emitted in CMAKE_BUILD_TYPE=RelWithDebInfo.
It is unclear why invoking watch_is_sentinel() before
buf_fix_count() would make the warning disappear.
