Commit graph

753 commits

Author SHA1 Message Date
Marko Mäkelä
28c89b7151 Merge 10.4 into 10.5 2019-12-16 07:47:17 +02:00
Vladislav Vaintroub
3304004a57 MDEV-21260 Innodb/Wjndows do not report error when trying open volumes on UNC paths
fil_node_t::find_metadata() tries to find out whether file
is on an SSD, and the disk sector size.
On Windows, it opens the corresponding volume for finding this data.

This does not go well, if datadir is on network path/UNC. The volume name
is invalid, CreateFile() function fails, and a cryptic (from the end user
perspective) error is reported. Like this

[ERROR] InnoDB: File \\.\\\workpc\work: 'CreateFile()' returned OS error 203.

The fix is not to report error if open volume failed, and the path was not
on fixed disk, i.e not on HDD or SSD. This is not a fatal error, there is
a fallback anyway.
2019-12-12 11:50:18 +01:00
Vladislav Vaintroub
66de4fef76 MDEV-16264 - some improvements
- wait notification, tpool_wait_begin/tpool_wait_end - to notify the
threadpool that current thread is going to wait

Use it to wait for IOs to complete and also when purge waits for workers.
2019-12-09 21:12:13 +01:00
Oleksandr Byelkin
a15234bf4b Merge branch '10.3' into 10.4 2019-12-09 15:09:41 +01:00
Marko Mäkelä
42a4ae54c2 MDEV-21225 Remove ut_align() and use aligned_malloc()
Before commit 90c52e5291 introduced
aligned_malloc(), InnoDB always used a pattern of over-allocating
memory and invoking ut_align() to guarantee the desired alignment.

It is cleaner to invoke aligned_malloc() and aligned_free() directly.

ut_align(): Remove. In assertions, ut_align_down() can be used instead.
2019-12-05 06:42:31 +02:00
Marko Mäkelä
504202bd7f MDEV-21216: Remove fsp_header_get_space_id()
The function fsp_header_get_space_id() returns ulint instead of
uint32_t, only to be able to complain that the two adjacent
tablespace ID fields in the page differ. Remove the function,
and merge the check to the callers.

Also, make some more use of aligned_malloc().
2019-12-04 20:01:04 +02:00
Oleksandr Byelkin
008ee867a4 Merge branch '10.2' into 10.3 2019-12-04 17:46:28 +01:00
Oleksandr Byelkin
f8b5e147da Merge branch '10.1' into 10.2 2019-12-03 14:45:06 +01:00
Eugene Kosov
543f22a2d0 check parameters of io_submit() and LinuxAIOHandler::reserve_slot()
Ensure that pointer to a buffer which is used for both reads and writes
is properly aligned.

I've intentially put permanent check right before io_submit() call instead
of the time of setting iocb date. I think that'll make check better.

LinuxAIOHandler::resever_slot(): check arguments alignment in debug builds
2019-12-03 14:01:11 +07:00
Jan Lindström
9d9a2253c6 Merge remote-tracking branch 10.2 into 10.3
Conflicts:
	mysql-test/suite/galera/t/galera_binlog_event_max_size_max-master.opt
	mysql-test/suite/innodb/r/innodb-mdev-7513.result
	mysql-test/suite/innodb/t/innodb-mdev-7513.test
	mysql-test/suite/wsrep/disabled.def
	storage/innobase/ibuf/ibuf0ibuf.cc
2019-12-02 14:35:10 +02:00
Eugene Kosov
33cf4da183 cleanup: replace exit(1) with abort() 2019-11-30 18:19:20 +07:00
Eugene Kosov
6fe2aae3ce InnoDB: log unsuccessful calls to pthread_attr_init() and pthread_create() before crash 2019-11-30 12:14:00 +07:00
Eugene Kosov
49ed1ae320 fix double io_destroy() + cleanup 2019-11-26 22:57:55 +07:00
Eugene Kosov
d30e51fafb cleanup Linux AIO
Replace all io_context* occurrences with io_context_t

Even in release mode die immediately when some io_* functions return
EINVAL. This always means some programming bug and it's better to fail fast.

LinuxAIOHandler::resubmit(): fix condition. Stop ignoring -1 return code which
corresponds to EPERM and io_submit() really can return this one.

Use io_destroy() to stop leaking io_context_t.

Make m_aio_ctx std::vector instead of C array. I think that internal check
for index overflow might be useful.

Add debug assertions for EFAULT because for me receiving it
looks like a programming bug.
2019-11-21 22:09:57 +07:00
Marko Mäkelä
15c7f684ec MDEV-16264: Minor cleanup
Add missing static qualifiers.
2019-11-15 22:54:42 +02:00
Marko Mäkelä
8040998624 MDEV-16264: Fix some white space 2019-11-15 19:55:13 +02:00
Vladislav Vaintroub
5e62b6a5e0 MDEV-16264 Use threadpool for Innodb background work.
Almost all threads have gone
- the "ticking" threads, that sleep a while then do some work)
(srv_monitor_thread, srv_error_monitor_thread, srv_master_thread)
were replaced with timers. Some timers are periodic,
e.g the "master" timer.

- The btr_defragment_thread is also replaced by a timer , which
reschedules it self when current defragment "item" needs throttling

- the buf_resize_thread and buf_dump_threads are substitutes with tasks
Ditto with page cleaner workers.

- purge workers threads are not tasks as well, and purge cleaner
coordinator is a combination of a task and timer.

- All AIO is outsourced to tpool, Innodb just calls thread_pool::submit_io()
and provides the callback.

- The srv_slot_t was removed, and innodb_debug_sync used in purge
is currently not working, and needs reimplementation.
2019-11-15 18:09:30 +01:00
Oleksandr Byelkin
3ad37ed0eb Merge 10.4 into 10.5 2019-11-07 08:52:30 +01:00
Marko Mäkelä
ec40980ddd Merge 10.3 into 10.4 2019-11-01 15:23:18 +02:00
Oleksandr Byelkin
55b2281a5d Merge branch '10.2' into 10.3 2019-10-31 10:58:06 +01:00
Marko Mäkelä
0ccfdc8eff Remove InnoDB wrappers of <string.h> functions
ut_strcmp(), ut_strcpy(), ut_strlen(), ut_memcpy(), ut_memcmp(),
ut_memmove(): Remove. Invoke the standard library functions directly.
2019-10-30 07:31:39 +02:00
Vladislav Vaintroub
2d82ae5ba3 MDEV-20825 : Innodb does not start if GetDiskFreeSpace() fails.
Ignore GetDiskFreeSpace() errors in os_file_get_status_win32
The call is only used to calculate filesystem block size, and this in
turn is only  shown in information_schema.sys_tablespaces.FS_BLOCK_SIZE.
There is no other use of this field, it does not affect any Innodb
functionality
2019-10-28 17:01:32 +00:00
Marko Mäkelä
624dd71b94 Merge 10.4 into 10.5 2019-08-13 18:57:00 +03:00
Marko Mäkelä
e9c1701e11 Merge 10.3 into 10.4 2019-07-25 18:42:06 +03:00
Marko Mäkelä
fdef9f9b89 Merge 10.2 into 10.3 2019-07-25 15:31:11 +03:00
Marko Mäkelä
7de38492fc After-merge fix: cmake -DPLUGIN_PERFSCHEMA=NO
An #include was forgotten in b6ac67389d
2019-07-25 13:34:31 +03:00
Marko Mäkelä
b6ac67389d Merge 10.1 into 10.2 2019-07-25 12:14:27 +03:00
Marko Mäkelä
0c7c61019d Remove the wrappers ut_time(), ut_difftime(), ib_time_t 2019-07-24 21:59:26 +03:00
Marko Mäkelä
10ee1b95b8 Remove ut_usectime(), ut_gettimeofday()
Replace ut_usectime() with my_interval_timer(),
which is equivalent, but monotonically counting nanoseconds
instead of counting the microseconds of real time.

os_event_wait_time_low(): Use my_hrtime() instead of ut_usectime().

FIXME: Set a clock attribute on the condition variable that allows
a monotonic clock to be chosen as the time base, so that the wait
is immune to adjustments of the system clock.
2019-07-24 21:59:26 +03:00
Vladislav Vaintroub
4513e73e0f Remove os_aio_simulated_put_read_threads_to_sleep()
According to the code, it was Windows specific "simulated AIO"
workaround. The simulated s not supported on Windows anymore.

Thus, remove the dead code
2019-07-04 10:27:11 +02:00
Vladislav Vaintroub
bd917e0811 Fix clang-cl warnings 2019-07-04 10:27:10 +02:00
Marko Mäkelä
412533b4a7 MDEV-18582: Extend SHOW STATUS LIKE 'Innodb_%'
Many InnoDB internal variables and counters were only exposed
in an unstructured fashion via SHOW ENGINE INNODB STATUS.

Expose more variables via SHOW STATUS. Many of these were
exported in XtraDB.

Also, introduce SHOW_SIZE_T and use the proper size for
exporting the InnoDB variables.

Remove some unnecessary indirection via export_vars, and
bind some variables directly.

dict_sys_t::rough_size(): Replaces dict_sys_get_size()
and includes the hash table sizes.

This is based on a contribution by Tony Liu from ServiceNow.
2019-07-04 00:09:16 +03:00
Marko Mäkelä
984d7100cd Merge 10.4 into 10.5 2019-06-13 18:36:09 +03:00
Marko Mäkelä
2fd82471ab Merge 10.3 into 10.4 2019-06-12 08:37:27 +03:00
Marko Mäkelä
b42dbdbccd Merge 10.2 into 10.3 2019-06-11 13:00:18 +03:00
Marko Mäkelä
cbac8f9351 MDEV-19725 Incorrect error handling in ALTER TABLE
Some I/O functions and macros that are declared in os0file.h used to
return a Boolean status code (nonzero on success). In MySQL 5.7, they
were changed to return dberr_t instead. Alas, in MariaDB Server 10.2,
some uses of functions were not adjusted to the changed return value.

Until MDEV-19231, the valid values of dberr_t were always nonzero.
This means that some code that was incorrectly checking for a zero
return value from the functions would never detect a failure.

After MDEV-19231, some tests for ALTER ONLINE TABLE would fail with
cmake -DPLUGIN_PERFSCHEMA=NO. It turned out that the wrappers
pfs_os_file_read_no_error_handling_int_fd_func() and
pfs_os_file_write_int_fd_func() were wrongly returning
bool instead of dberr_t. Also the callers of these functions were
wrongly expecting bool (nonzero on success) instead of dberr_t.

This mistake had been made when the addition of these functions was
merged from MySQL 5.6.36 and 5.7.18 into MariaDB Server 10.2.7.

This fix also reverts commit 40becbc3c7
which attempted to work around the problem.
2019-06-10 18:15:25 +03:00
Sergey Vojtovich
5f5a0b3bb6 MDEV-16548 - Innodb fails to start on older kernels that don't support F_DUPFD_CLOEXEC
InnoDB duplicates file descriptor returned by create_temp_file() to
workaround further inconsistent use of this descriptor.

Use mysys file descriptors consistently for innobase_mysql_tmpfile(NULL).
Mostly close it by appropriate mysys wrappers.
2019-05-24 13:21:09 +04:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Vicențiu Ciorbaru
c0ac0b8860 Update FSF address 2019-05-11 19:25:02 +03:00
Vladislav Vaintroub
60bd353bdf Fixes for atomic writes on Windows.
Windows does atomic writes, as long as they are aligned and multiple
of sector size. this is documented in MSDN.

Fix innodb.doublewrite test to always use doublewrite buffer,
(even if atomic writes are autodetected)
2019-05-06 11:32:17 +00:00
Vladislav Vaintroub
59a266a9f9 MDEV-17380: fix incorrect #ifdef 2019-05-06 11:31:51 +00:00
Vladislav Vaintroub
c477623f04 MDEV-19388 Improve SSD detection on Windows
Fallback to detecting if TRIM is enabled, if we cannot determine
seek penalty.
2019-05-06 10:15:46 +00:00
Marko Mäkelä
d3dcec5d65 Merge 10.3 into 10.4 2019-05-05 15:06:44 +03:00
Marko Mäkelä
b132b8895e Merge 10.3 into 10.4 2019-05-05 10:23:14 +03:00
Marko Mäkelä
4d59f45260 Merge 10.2 into 10.3 2019-04-27 20:41:31 +03:00
Sachin Agarwal
06ec56f579 Bug #27850600 INNODB ASYNC IO ERROR HANDLING IN IO_EVENT
Problem:
io_getevents() - read asynchronous I/O events from the completion
queue. For each IO event, the res field in io_event tells whether IO
event is succeeded or not. To see if the IO actually succeeded we
always need to check event.res (negative=error,
positive=bytesread/written).
LinuxAIOHandler::collect() doesn't check event.res value for each event.
which leads to incorrect value in n_bytes for IO context (or IO Slot).

Fix:
Added a check for event.res negative value.

RB: 20871
Reviewed by : annamalai.gurusami@oracle.com
2019-04-26 17:40:20 +03:00
Marko Mäkelä
edd1a53a55 Merge 10.3 into 10.4 2019-04-08 22:00:07 +03:00
Marko Mäkelä
9ba0865b87 Merge 10.2 into 10.3 2019-04-08 21:38:13 +03:00