Commit graph

193999 commits

Author SHA1 Message Date
Vladislav Vaintroub
b4074069b2 MDEV-26657 : Initialize some fields in create_background_thd()
Avoid reading uninitialized memory by  thd_get_error_context_description().
Note, that THD::real_id can't be initialized at this stage, so it will be zeroed.
2021-09-21 19:14:07 +02:00
Monty
f2021b5ee4 Fixed random failure of main.truncate_notembedded
The test depends on how the server allocates memory and may fail randomly.
Fixed by accepting that TRUNCATE may work in some cases (happened to me)
2021-09-21 15:06:22 +03:00
Monty
d6bddfca22 Updated main.alias test to fix max_length
The test will work after libmariadb has been updated to return correct
max_length for prepared statements
2021-09-21 15:06:22 +03:00
Monty
a584117c0c Fixed max length calculation for embedded server
The old code did not take into account unsigned numbers when calculating
max_lengths of fields.
2021-09-21 15:06:22 +03:00
Alexey Bychko
25d6bbcd51 MDEV-23506 mariadb-connector-c-devel package from standard RHEL 8 repo conflicts with MariaDB's packages
added alternative name for MariaDB-devel package to replace
mariadb-connector-c-devel from RHEL 8 distribution.
this patch is for 10.3+ on RHEL/Centos 8
2021-09-21 14:44:39 +07:00
Julius Goryavsky
f4d6d01782 MDEV-26441: Linux-dependent construct in SST scripts
SST scripts currently use Linux-specific construction
to create a temporary directory if the path prefix for
that directory is specified by the user. This does not
work with FreeBSD. This commit adds support for FreeBSD.

No separate test required.
2021-09-20 12:10:53 +02:00
Marko Mäkelä
3209bc667f MDEV-26636: InnoDB defragmentation statistics cause races on TEMPORARY TABLE
btr_defragment_save_defrag_stats_if_needed(): Do not save
defragmentation statistics for temporary tables.
They are exempt of defragmentation anyway
(ha_innobase::optimize() never invokes defragmentation for them),
and the user-visible names are not available inside InnoDB.

Furthermore, InnoDB assumes that temporary tables are never accessed
by other threads than the one that handles the session with which
the temporary table is associated with.

Furthermore, we simplify the test innodb.innodb_defrag_stats
and include a test case that demonstrates that defragmentation
statistics are no longer being saved for temporary tables.
2021-09-18 15:47:52 +03:00
Marko Mäkelä
b740b2356d MDEV-25919 fixup: Acquire MDL also in defragmentation
dict_stats_process_entry_from_defrag_pool(): Acquire MDL on the table
for which we are invoking dict_stats_save_defrag_stats(), to avoid
any race condition with DROP TABLE or similar operations.
2021-09-18 14:39:32 +03:00
Marko Mäkelä
56843d62f9 MDEV-24258 fixup: Correct a condition
dict_stats_process_entry_from_defrag_pool(): Restore a condition as it was
before commit 82b7c561b7.
2021-09-18 14:19:55 +03:00
Marko Mäkelä
ae8c8d8874 Merge 10.5 into 10.6 2021-09-17 20:07:38 +03:00
Marko Mäkelä
699de65d5e Merge 10.4 into 10.5 2021-09-17 19:57:13 +03:00
Thirunarayanan Balathandayuthapani
496d3dded4 MDEV-15675 encryption.innodb_encryption failed in buildbot with timeout
Test case fail to include undo tablespace while waiting for the
encryption thread to encrypt all existing tablespace
2021-09-17 19:37:57 +05:30
Monty
03a10706ec Fixed alias.test to also works with ps
The issue is that max_length for prepared statements are different from
normal queries, which can optimize the max_length based on the result
length.
2021-09-17 16:07:39 +03:00
Marko Mäkelä
1e9c922fa7 MDEV-26623 Possible race condition between statistics and bulk insert
When computing statistics, let us play it safe and check whether
an insert into an empty table is in progress, once we have acquired
the root page latch. If yes, let us pretend that the table is empty,
just like MVCC reads would do.

It is unlikely that this race condition could lead into any crashes
before MDEV-24621, because the index tree structure should be protected
by the page latches. But, at least we can avoid some busy work and
return earlier.

As part of this, some code that is only used for statistics calculation
is being moved into static functions in that compilation unit.
2021-09-17 14:40:41 +03:00
Marko Mäkelä
1a4a7dddb6 Cleanup: Make btr_root_block_get() more robust
btr_root_block_get(): Check for index->page == FIL_NULL.

btr_root_get(): Declare static. Other callers can invoke
btr_root_block_get() directly.

btr_get_size(): Remove conditions that are checked in
btr_root_block_get().
2021-09-17 14:40:41 +03:00
Krunal Bauskar
48bbc44733 MDEV-26609 : Avoid deriving ELEMENT_PER_LATCH from cacheline
* buffer pool has latches that protect access to pages.

* there is a latch per N pages.
  (check page_hash_table for more details)

* N is calculated based on the cacheline size.

* for example: if cacheline size is
  : 64 then 7 pages pointers + 1 latch can be hosted on the same cacheline
  : 128 then 15 pages pointers + 1 latch can be hosted on the same cacheline

* arm generally have wider cacheline so with arm 1 latch is used
  to access 15 pages vs with x86 1 latch is used to access 7 pages.
  Naturally, the contention is more with arm case.

* said patch help relax this contention by limiting the elements
  per cacheline to 7 (+ 1 latch slot).
  for wider-cacheline (say 128), the remaining 8 slots are kept empty.
  this ensures there are no 2 latches on the same cacheline to avoid
  latch level contention.

Based on suggestion from Marko, the same logic is now extended to
lock_sys_t::hash_table.
2021-09-17 11:58:49 +03:00
Julius Goryavsky
a2e55131c6 MDEV-19950 addendum: galera_ssl_upgrade removed from the list of disabled tests and adapted for 10.4+ 2021-09-17 09:26:35 +02:00
Jan Lindström
d3b35598fc MDEV-26053 : TRUNCATE on table with Foreign Key Constraint no longer replicated to other nodes
Problem was that there was extra condition !thd->lex->no_write_to_binlog
before call to begin TOI. It seems that this variable is not initialized.
TRUNCATE does not support [NO_WRITE_TO_BINLOG | LOCAL] keywords, thus
we should not check this condition. All this was hidden in a macro,
so I decided to remove those macros that were used only a few places
with actual function calls.
2021-09-17 07:18:37 +03:00
Marko Mäkelä
106b16a5af MDEV-26356 fixup: integer type mismatch on 32-bit 2021-09-17 07:01:01 +03:00
Marko Mäkelä
9d8e83b676 MDEV-26356 fixup: Adjust innodb_max_purge_lag_wait
innodb_max_purge_lag_wait_update(): To align with
purge_coordinator_state::refresh(), we must trigger a page flush batch
if the last log checkpoint is too old.

This was caught in a hang of the test innodb_gis.rtree_compress.
2021-09-16 20:30:08 +03:00
Marko Mäkelä
03c09837fc Merge 10.5 into 10.6 2021-09-16 20:17:12 +03:00
Marko Mäkelä
c430aa72ab MDEV-26626 InnoDB fails to advance the log checkpoint
buf_flush_page_cleaner(): Always try to advance the log checkpoint,
even when no pages were flushed during the latest batch.
Maybe, since the previous batch, there was an LRU flush that
removed the last dirty pages.

Failure to advance the log checkpoint will cause unnecessary work
in Mariabackup and on crash recovery.
2021-09-16 20:10:42 +03:00
Oleksandr Byelkin
dce490e9d4 Give less memory to get reliable error. 2021-09-16 17:29:37 +02:00
Monty
65cce297be Updated rocksdb test result
This was required as I added a new error code to my_base.h and rocksdb
is adding it's own errors after the last official one

Updated result file also for index_merge_rocksdb2. This is a big test
and we have probably not before noticed that some optimizer changes
caused a difference.
2021-09-16 14:06:29 +03:00
Daniel Black
dabc3329c3 MDEV-26622: man mysqldump - insert-ignore not insert-into
Thanks Karl Levik
2021-09-16 20:52:31 +10:00
Eugene Kosov
5b0a76078a MDEV-26621 assertion failue "index->table->persistent_autoinc" in /storage/innobase/btr/btr0btr.cc during IMPORT
dict_index_t::clear_instant_alter(): when searhing for an AUTO_INCREMENT column
don't skip the beginning of the list because the field can be at the beginning of the list
2021-09-16 16:52:20 +06:00
Monty
0d47945b58 Fixed bug in aria_chk that overwrote sort_buffer_length
This bug happens when one runs aria_chk on multiple tables. It does not
affect REPAIR TABLE.
aria_chk tries to optimize the sort buffer size to minimize memory usage
when used with small tables. The bug was that the adjusted value was
used as a base for the next table, which could cause problems.
2021-09-15 21:21:03 +03:00
Monty
b4f24c745a Merge branch '10.4' into 10.5
Fixed also an error in suite/perfschema/t/transaction_nested_events-master.opt
2021-09-15 20:23:07 +03:00
Monty
f03fee06b0 Improve error messages from Aria
- Error on commit now returns HA_ERR_COMMIT_ERROR instead of
  HA_ERR_INTERNAL_ERROR
- If checkpoint fails, it will now print out where it failed.
2021-09-15 19:27:34 +03:00
Monty
6be0ddae5e Fixed compiler warnings in CONNECT 2021-09-15 19:16:37 +03:00
Vladislav Vaintroub
b1351c1594 MDEV-26574 An improper locking bug due to unreleased lock in the ds_xbstream.cc
release lock in all as cases n xbstream_open, also fix the case where malloc would return NULL.
2021-09-15 14:55:45 +02:00
Vladislav Vaintroub
8937762ead MDEV-26573 : A static analyzer warning about ds_archive.cc
This file had not been compiled for long time.
Remove this from the tree.
2021-09-15 14:19:24 +02:00
Monty
689b8d060a MDEV-23519 Protocol packet - "Original Name" info is showing alias name,
instead of original name of the column

When doing refactoring of temporary table field creation a mistake was
done when copying the column name when creating internal temporary tables.
For internal temporary tables we should use the original field name, not
the item name (= alias).
2021-09-14 20:14:04 +03:00
Marko Mäkelä
ea52a3eb97 MDEV-26356 Adaptive purge scheduling based on redo log fill factor
This should be equivalent to pull request #1889 by Krunal Bauskar.

The existing logic in purge_coordinator_state::do_purge()
activates a number of the configured innodb_purge_threads
based on the history list length. Activating more purge worker
tasks should shrink the history list faster. But, more purge
workers will also generate more redo log, which may slow down
writes by user connections.

row_purge_parse_undo_rec(): Revert the work-around that was added in
commit 4690442411.

purge_coordinator_state: Keep track of the redo log fill factor
(how big percentage of innodb_log_file_size is being occupied by
log records that were generated since the latest checkpoint).
If the redo log is getting full, log checkpoints will be triggered
more frequently, and user threads may end up waiting in
log_free_check(). We try to reduce purge-induced jitter in overall
throughput by throttling down the active number of purge tasks as
the log checkpoint age is approaching the log size (in other words,
the redo log fill factor is approaching 100%).
2021-09-14 19:06:05 +03:00
Marko Mäkelä
717a32154c MDEV-26356 preparation: Refactor purge_state
purge_coordinator_timer_callback(): Remove. We will have
purge_coordinator_timer invoke purge_coordinator_callback()
directly.

srv_master_callback(): Invoke srv_wake_purge_thread_if_not_active()
instead of purge_coordinator_timer_callback(). That is, we will
trigger purge_coordinator_callback() once per second if there is
any work to be done.

purge_state::do_purge(): Replaces srv_do_purge(),
purge_coordinator_callback_low(), and
purge_coordinator_timer_callback(). The static variables
inside srv_do_purge() were moved to purge_state data members.
2021-09-14 19:05:05 +03:00
Marko Mäkelä
03e4cb2484 MDEV-24512 fixup: Remove after_task_callback
In commit ff5d306e29 we removed
dbug_after_task_callback but forgot to revert the rest of
commit bada05a883.
2021-09-14 16:23:23 +03:00
Daniel Black
adaf0dde7f MDEV-26601: mysys - O_TMPFILE ^ O_CREAT
Thanks to Fabian Vogt for noticing the mutual exclusions
of these open flags on tmpfs caused by mariadb opening it
incorrectly.

As such we clear the O_CREAT flag while opening it as O_TMPFILE.
2021-09-14 21:06:34 +10:00
Daniele Sciascia
5527fc5861 MDEV-21613 Failed to open table mysql.wsrep_streaming_log for writing
Fix sporadic failure for MTR test galera_sr.GCF-1018B. The test
sometimes fails due to an error that is logged to the error log
unnecessarily.
A deterministic test case (included in this patch) shows that the
error is loggen when a transaction is BF aborted right before  it
opens the streaming log table to perform fragment removal. When that
happens, the attempt to open the table fails and consequently an error
is logged. There is no need to log this error, as an ER_LOCK_DEADLOCK
error is returned to the client.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2021-09-14 11:38:03 +03:00
Otto Kekäläinen
07abcb5045 Deb: Fix Gitlab-CI/Salsa-CI builds failures
The debian/salsa-ci.yml used to work also on upstream MariaDB.org branches,
but has recently regressed and several jobs stopped working. These fixes
are necessary to get it working again.

* Partially revert 8642f592 that never worked, as MariaDB 10.2 does not
  have a mysql.global table nor a mariadb.sys user. Those features weren't
  introduced until MariaDB 10.4.

* Partially revert 0268b871 as we don't want ColumnStore as part of the
  native Debian build. It should build only when the build is triggered
  via autobake-deb.sh (MariaDB.org builds).

* Adjust salsa-ci.yml to cope with various Stretch to Sid upgrade issues
  and remove the legacy mariadb-connector-c job completely as that package
  hasn't been around for years anymore.

* Extend Lintian overrides to be otherwise Lintian clean
2021-09-13 17:34:55 -07:00
Marko Mäkelä
f6717c4af6 MDEV-24258 fixup: Do not update dict_table_t::n_ref_count
row_purge_remove_clust_if_poss_low(): The table identified by
SYS_INDEXES.TABLE_ID is protected by exclusive dict_sys.latch.
There is no need to touch its reference count.
2021-09-13 11:55:14 +03:00
Thirunarayanan Balathandayuthapani
696de6d06c MDEV-25702 Auxiliary FTS table evicts during optimize table
InnoDB could evict the fts auxiliary table in
row_fts_merge_insert(). So bulk insert could be
dealing with garbage FTS auxiliary table.Patch
should delay closing the table in row_fts_merge_insert().
2021-09-13 12:39:17 +05:30
Sergei Golubchik
1a6c130c4f perfschema: use correct type for left shifts
set_item() uses 1UL << bit, so is_set_item() must do the same.

This fixes sporadic perfschema.show_aggregate failures
(sporadic, because `bit` is the thread id, so depending on how many
tests were run before perfschema.show_aggregate it can be above or
below 32).
2021-09-12 15:42:49 +02:00
Vladislav Vaintroub
329d370f2d Merge branch '10.5' into 10.6
# Conflicts:
#	appveyor.yml
2021-09-11 19:05:25 +02:00
Vladislav Vaintroub
b112c9dfaa Fix Connect build with MSVC+Ninja 2021-09-11 18:32:07 +02:00
Vladislav Vaintroub
42e9506ce9 Merge branch '10.4' into 10.5 2021-09-11 17:19:36 +02:00
Vladislav Vaintroub
74368a1df8 Merge branch '10.3' into 10.4 2021-09-11 17:09:11 +02:00
Marko Mäkelä
15139964d5 Merge 10.5 into 10.6 2021-09-11 17:55:27 +03:00
Vladislav Vaintroub
e42da92265 Use mariadb- named targets for minbuild 2021-09-11 16:48:33 +02:00
Vladislav Vaintroub
cef656b11c Fix Windows warnings and tests for -DPLUGIN_PERFSCHEMA=NO 2021-09-11 16:47:59 +02:00
Vladislav Vaintroub
c3341f8440 Merge branch '10.4' into 10.5
# Conflicts:
#	appveyor.yml
2021-09-11 16:40:08 +02:00