Commit graph

191,256 commits

Author SHA1 Message Date
Monty
c8992fc35b Trivial cleanups, no logic changes
- Fold long comment rows and updated comments
- Moved one private function in class Item_func_rand among other private
  functions
2020-11-26 19:13:37 +02:00
Monty
3d56bea3ef Allow field_name NOT NULL ENABLED
This is for Oracle compatiblity. ENABLED is in Oracle the default case
and just ensures that the NOT NULL constraints will be tested, which is
also default in MariaDB
2020-11-26 19:13:37 +02:00
Monty
55b2788800 Fixed length estimate for REPLACE() 2020-11-26 19:13:37 +02:00
Marko Mäkelä
657fcdf430 MDEV-24280 InnoDB triggers too many independent periodic tasks
A side effect of MDEV-16264 is that a large number of threads will
be created at server startup, to be destroyed after a minute or two.

One source of such thread creation is srv_start_periodic_timer().
InnoDB is creating 3 periodic tasks: srv_master_callback (1Hz)
srv_error_monitor_task (1Hz), and srv_monitor_task (0.2Hz).

It appears that we can merge srv_error_monitor_task and srv_monitor_task
and have them invoked 4 times per minute (every 15 seconds). This will
affect our ability to enforce innodb_fatal_semaphore_wait_threshold and
some computations around BUF_LRU_STAT_N_INTERVAL.

We could remove srv_master_callback along with the DROP TABLE queue
at some point of time in the future. We must keep it independent
of the innodb_fatal_semaphore_wait_threshold detection, because
the background DROP TABLE queue could get stuck due to dict_sys
being locked by another thread. For now, srv_master_callback
must be invoked once per second, so that
innodb_flush_log_at_timeout=1 can work.

BUF_LRU_STAT_N_INTERVAL: Reduce the precision and extend the time
from 50*1 second to 4*15 seconds.

srv_error_monitor_timer: Remove.

MAX_MUTEX_NOWAIT: Increase from 20*1 second to 2*15 seconds.

srv_refresh_innodb_monitor_stats(): Avoid a repeated call to time(NULL).
Change the interval to less than 60 seconds.

srv_monitor(): Renamed from srv_monitor_task.

srv_monitor_task(): Renamed from srv_error_monitor_task().
Invoked only once in 15 seconds. Invoke also srv_monitor().
Increase the fatal_cnt threshold from 10*1 second to 1*15 seconds.

sync_array_print_long_waits_low(): Invoke time(NULL) only once.
Remove a bogus message about printouts for 30 seconds. Those
printouts were effectively already disabled in MDEV-16264
(commit 5e62b6a5e0).
2020-11-25 16:54:00 +02:00
Marko Mäkelä
7b1252c03d MDEV-24278 InnoDB page cleaner keeps waking up on idle server
The purpose of the InnoDB page cleaner subsystem is to write out
modified pages from the buffer pool to data files. When the
innodb_max_dirty_pages_pct_lwm is not exceeded or
innodb_adaptive_flushing=ON decides not to write out anything,
the page cleaner should keep sleeping indefinitely until the state
of the system changes: a dirty page is added to the buffer pool such
that the page cleaner would no longer be idle.

buf_flush_page_cleaner(): Explicitly note when the page cleaner is idle.
When that happens, use mysql_cond_wait() instead of mysql_cond_timedwait().

buf_flush_insert_into_flush_list(): Wake up the page cleaner if needed.

innodb_max_dirty_pages_pct_update(),
innodb_max_dirty_pages_pct_lwm_update():
Wake up the page cleaner just in case.

Note: buf_flush_ahead(), buf_flush_wait_flushed() and shutdown are
already waking up the page cleaner thread.
2020-11-25 16:09:47 +02:00
Marko Mäkelä
f693b72547 MDEV-24270: Clarify some comments 2020-11-25 16:08:26 +02:00
Vladislav Vaintroub
2de95f7a1e Fix misspelling.
Kudos to Marko for finding.
2020-11-25 13:07:49 +01:00
Vladislav Vaintroub
af98fddc2b Cleanup. Remove obsolete comment 2020-11-25 13:07:08 +01:00
Vladislav Vaintroub
c130c60b2b Cleanup. Provide accurate comment on my_getevents(). 2020-11-25 13:07:08 +01:00
Vladislav Vaintroub
78df9e37a6 Partially Revert "MDEV-24270: Collect multiple completed events at a time"
This partially reverts commit 6479006e14.

Remove the constant tpool::aio::N_PENDING, which has no
intrinsic meaning for the tpool.
2020-11-25 13:07:08 +01:00
Marko Mäkelä
3dfeae0e22 Cleanup: Fix Intel compiler warnings about sign conversions 2020-11-25 11:32:49 +02:00
Marko Mäkelä
4a22056c97 Cleanup: Remove redundant nonnull attributes 2020-11-25 11:28:26 +02:00
Marko Mäkelä
6479006e14 MDEV-24270: Collect multiple completed events at a time
tpool::aio::N_PENDING: Replaces OS_AIO_N_PENDING_IOS_PER_THREAD.
This limits two similar things: the number of outstanding requests
that a thread may io_submit(), and the number of completed requests
collected at a time by io_getevents().
2020-11-25 09:42:38 +02:00
Marko Mäkelä
7a9405e3dc MDEV-24270 Misuse of io_getevents() causes wake-ups at least twice per second
In the asynchronous I/O interface, InnoDB is invoking io_getevents()
with a timeout value of half a second, and requesting exactly 1 event
at a time.

The reason to have such a short timeout is to facilitate shutdown.

We can do better: Use an infinite timeout, wait for a larger maximum
number of events. On shutdown, we will invoke io_destroy(), which
should lead to the io_getevents system call reporting EINVAL.

my_getevents(): Reimplement the libaio io_getevents() by only invoking
the system call. The library implementation would try to elide the
system call and return 0 immediately if aio_ring_is_empty() holds.
Here, we do want a blocking system call, not 100% CPU usage. Neither
do we want the aio_ring_is_empty() trigger SIGSEGV because it is
dereferencing some memory that was freed by io_destroy().
2020-11-25 09:40:12 +02:00
Marko Mäkelä
1b12e251cd MDEV-24271 rw_lock::read_lock_yield() may cause writer starvation
The greedy fetch_add(1) approach of read_trylock() may cause
starvation of a waiting write lock request. Let us use a
compare-and-swap for the read lock acquisition in order to
guarantee the progress of writers.
2020-11-24 14:01:00 +02:00
Marko Mäkelä
dcdc8c3506 MDEV-21534 fixup: Remove HAVE_IB_LINUX_FUTEX
Since commit 30ea63b7d2
we actually depend on futex on Linux. Also, we depend on
std::atomic for even longer.
2020-11-23 15:45:03 +02:00
Marko Mäkelä
1e5d989d2a MDEV-24167: Remove PFS instrumentation of buf_block_t
We always defined PFS_SKIP_BUFFER_MUTEX_RWLOCK, that is,
the latches of the buffer pool blocks were never instrumented
in PERFORMANCE_SCHEMA.

For some reason, the debug_latch (which enforce proper usage of
buffer-fixing in debug builds) was instrumented.
2020-11-20 08:55:41 +02:00
Marko Mäkelä
156cb94b4b MDEV-22871 fixup: Relax a debug assertion
In commit bf3c862faa we introduced
an assertion that may dereference a null pointer.

This regression was caught by running the following:
./mtr --parallel=auto --suite=innodb \
--mysqld=--loose-innodb-adaptive-hash-index

The adaptive hash index is disabled by default since
commit 88cdfc5c7d (MDEV-20487)
and hence the problem was not caught earlier.
2020-11-20 08:49:46 +02:00
Marko Mäkelä
3c8ecb5bbd Run innodb_wl6326_big only in debug builds
The test seems to deterministically fail on RelWithDebInfo builds
due to a timeout in wait_condition.inc.

According to Matthias Leich (the original author of the test),
the failure rate would reduce if we disabled the purge of
transaction history by setting innodb_force_recovery=2.

For now, let us run this stress test on debug builds only.
2020-11-20 08:06:24 +02:00
Marko Mäkelä
8ac19be8a2 Cleanup: Fix build problems with the Intel compiler
fil_space_t::flush_low(): Define and declare without inline.

ut_is_2pow(): Remove UNIV_LIKELY. This is almost exclusively
used in debug assertions. UNIV_LIKELY is not compatible with
static_assert in some compilers.
2020-11-20 07:59:46 +02:00
Marko Mäkelä
9c455945f5 MDEV-21534 fixup: Use a compile-time constant 2020-11-20 07:59:41 +02:00
Roman Nozdrin
a16e3c326d Update MCS to resolve libmarias3 compulation for centos74-amd64-debug
pipeline in community BB

Fix for rebuild from source step

Disable MCS on i386|i686 platforms

This patch puts MCS debian packaging files and part of debian/control
into the engine directory
2020-11-19 13:52:33 +00:00
Daniel Black
3b486c28f7 MDEV-24125: linux large pages, linux/mman.h needed
Centos/RHEL7 have the MAP_HUGE_SHIFT constant
defined in linux/mman.h which needed to get included.
2020-11-19 16:30:17 +11:00
Marko Mäkelä
33d41167c5 MDEV-24224 Gap lock on delete in 10.5 using READ COMMITTED
When MDEV-19544 (commit 1a6f470464)
simplified the initialization of the local variable
set_also_gap_locks, an inadvertent change was included.
Essentially, all code branches that are executed when
set_also_gap_locks hold must also ensure that
trx->isolation_level > TRX_ISO_READ_COMMITTED holds.
This was being violated in a few code paths.

It turns out that there is an even simpler fix: Remove the test
of thd_is_select() completely. In that way, the first part of
UPDATE or DELETE should work exactly like SELECT...FOR UPDATE.

thd_is_select(): Remove.
2020-11-18 10:27:18 +02:00
Marko Mäkelä
fabdad6857 Merge 10.4 into 10.5 2020-11-17 18:15:13 +02:00
Marko Mäkelä
bbf0b55c17 Work around MDEV-24232: Skip perfschema.nesting if WITH_WSREP=OFF 2020-11-17 18:13:47 +02:00
Marko Mäkelä
83a55670a6 MDEV-24188 fixup: Simplify the wait loop
Starting with commit 7cffb5f6e8 (MDEV-23399)
the function buf_flush_page() will first acquire block->lock and only
after that invoke set_io_fix(). Before that, it was possible to reach
a livelock between buf_page_create() and buf_flush_page().

buf_page_create(): Directly try acquiring the exclusive page latch
without checking whether the page is io-fixed or buffer-fixed.
(As a matter of fact, the have_x_latch() check is not strictly necessary,
because we still support recursive X-latches.)
In case of a latch conflict, wait while allowing buf_page_write_complete()
to acquire buf_pool.mutex and release the block->lock.

An attempt to wait for exclusive block->lock while holding buf_pool.mutex
would lead to a hang in the tests parts.part_supported_sql_func_innodb
and stress.ddl_innodb, due to a deadlock between buf_page_write_complete()
and buf_page_create().

Similarly, in case of an I/O fixed compressed-only
ROW_FORMAT=COMPRESSED page, we will sleep before retrying.

In both cases, we will sleep for 1ms or until a flush batch is completed.
2020-11-17 17:56:50 +02:00
Dmitry Shulga
796f708f85 MDEV-24115 Fix -Wconversion in Timeval::Timeval() on Mac OS X
The data member tv_usec of the struct timeval is declared as suseconds_t
on MacOS. Size of suseconds_t is 4 bytes. On the other hand, size of ulong
is 8 bytes on 64-bit MacOS, so attempt to assign a value of wider type
(usec) to a value (tv_usec) of narrower type leads to error.
2020-11-17 19:23:33 +07:00
Marko Mäkelä
f0c9903795 Merge 10.3 into 10.4 2020-11-17 13:56:20 +02:00
Daniele Sciascia
694926a4f7 Fix suppression in MTR test galera_3nodes.inconsistency_shutdown 2020-11-17 13:00:44 +02:00
Sujatha
c815ffb975 MDEV-23610: Slave user can't run "SHOW SLAVE STATUS" anymore after upgrade to 10.5, mysql_upgrade should take of that
Post push fix. Update version to 10.5.8.
2020-11-17 16:08:32 +05:30
Daniel Black
7f30a5c423 MDEV-24125: allow compile on Linux headers < 3.8
This allows MariaDB to compile on old (limits to >2.6.32)
linux kernel versions.

This warns that attempts to use large pages will rely on
implict kernel determination.
2020-11-17 07:53:55 +11:00
Daniel Black
8cc5d2845c MDEV-24125: linux large pages - Revert "Fixed centos 6 build failure"
This reverts commit 6cf8f05fd9.

Original patch assumed that MAP_HUGETLB as consistent across
achitectures which isn't the case. Defining it unconditionally
broke large pages on every achitecutre where the value differed
from x86_64.

With the EOL for Centos/RHEL6 announced in 10.5.7, <3.8 linux
kernels are no longer supported.
2020-11-17 07:53:55 +11:00
Daniel Black
0fc0eb1e5b MDEV-24124: main.drop test - mulitarch/os error messages
Account for variety of mips, hppa, solaris and other messages.

Copied from rpl.rpl_drop_db test.
2020-11-17 07:41:57 +11:00
Monty
eae9311fa2 Do not run maria.repair with --embedded as memory usage is different 2020-11-16 13:23:39 +02:00
Daniele Sciascia
1ae7809a7c Restore autoincrement offset in MTR test MDEV-24063 2020-11-16 12:32:08 +02:00
Sujatha
2b347e9fc5 MDEV-23610: Slave user can't run "SHOW SLAVE STATUS" anymore after upgrade to 10.5, mysql_upgrade should take of that
Fixing a post push test issue.
2020-11-16 14:49:46 +05:30
Sujatha
6da68049b5 MDEV-23610: Slave user can't run "SHOW SLAVE STATUS" anymore after upgrade to 10.5, mysql_upgrade should take of that
Add a new privilege "SLAVE MONITOR" which will grant user the permission
to execute "SHOW SLAVE STATUS" and "SHOW RELAYLOG EVENTS" commands.

SHOW SLAVE STATUS requires either SLAVE MONITOR/SUPER
SHOW RELAYLOG EVENTS requires SLAVE MONITOR privilege.
2020-11-16 14:31:44 +05:30
Roman Nozdrin
1edd224372 This patch puts MCS debian packaging files and part of debian/control
into the engine directory
2020-11-14 19:09:34 +00:00
Oleksandr Byelkin
81b9c78500 MDEV-24098: 10.5 followup
remove version data from the test output
2020-11-14 12:28:28 +01:00
Oleksandr Byelkin
6daf6bbcfe Merge branch '10.4' into 10.5 2020-11-14 10:15:54 +01:00
Oleksandr Byelkin
1bebc8de5d Merge branch '10.3' into 10.4 2020-11-14 10:05:23 +01:00
Oleksandr Byelkin
a00e21c03d Merge branch '10.2' into 10.3 2020-11-14 09:57:22 +01:00
Oleksandr Byelkin
c8be6aafb9 Fix to make it compiling on new ubuntu. 2020-11-14 09:55:09 +01:00
Marko Mäkelä
e8f8992801 MDEV-24188: Merge 10.4 into 10.5 2020-11-13 22:06:50 +02:00
Marko Mäkelä
ca331bdcda MDEV-23619: Merge 10.4 into 10.5 2020-11-13 22:06:05 +02:00
Marko Mäkelä
d7a5824899 Merge 10.4 into 10.5 2020-11-13 21:54:21 +02:00
Marko Mäkelä
749ecedfec MDEV-24188: Merge 10.3 into 10.4 2020-11-13 20:45:28 +02:00
Marko Mäkelä
b63dc3f370 MDEV-23619: Merge 10.3 into 10.4 2020-11-13 20:43:54 +02:00
Marko Mäkelä
f9f2f37495 MDEV-24188: Merge 10.2 into 10.3 2020-11-13 20:41:48 +02:00