Commit graph

187815 commits

Author SHA1 Message Date
Vladislav Vaintroub
ad17c98dd5 MDEV-16264 - prerequisite patch, periodic thr_timer
Threadpool will need a functionality for periodic thr_timer
(the threadpool maintainence task is a timer that runs periodically).

Also increase the stack size for the timer thread, 8k won't be enough.
2019-11-15 16:50:22 +01:00
Marko Mäkelä
786b004972 Cleanup: More use of mtr_memo_type_t 2019-11-15 14:55:38 +02:00
Marko Mäkelä
ae90f8431b Merge 10.4 into 10.5 2019-11-14 14:49:20 +02:00
Marko Mäkelä
89ae01fd00 Merge 10.3 into 10.4 2019-11-14 13:23:36 +02:00
Marko Mäkelä
746ee78535 MDEV-20949: Merge 10.3 into 10.4 2019-11-14 13:22:29 +02:00
Marko Mäkelä
3bee95d769 Merge 10.3 into 10.4 2019-11-14 13:20:57 +02:00
Jean Weisbuch
c454b8964c MDEV-4476: mytop tracker
- Added dynamic column width fro `user` and `db` columns
- Added option `%` to hide the progress
- Follow up changes from Ubuntu
- Take `INFORMATION_SCHEMA.PROCESSLIST` into account to determine
  progress of the `actual stage`
- Indentation fixes

Closes #215
2019-11-14 02:31:43 -08:00
Marko Mäkelä
3d4a801533 MDEV-12353 preparation: Replace mtr_x_lock() and friends
Apart from page latches (buf_block_t::lock), mini-transactions
are keeping track of at most one dict_index_t::lock and
fil_space_t::latch at a time, and in a rare case, purge_sys.latch.

Let us introduce interfaces for acquiring an index latch
or a tablespace latch.

In a later version, we may want to introduce mtr_t members
for holding a latched dict_index_t* and fil_space_t*,
and replace the remaining use of mtr_t::m_memo
with std::set<buf_block_t*> or with a map<buf_block_t*,byte*>
pointing to log records.
2019-11-14 11:40:33 +02:00
Marko Mäkelä
4ded5fb9ac MDEV-20949: Merge 10.2 into 10.3
In the test innodb.instant_alter,4k we would be flagging an error
for too large row size. That error was previously only being reported
if the table was being rebuilt. Thus, this merge is fixing a small
omission in MDEV-11369 (instant ADD COLUMN).
2019-11-14 11:26:49 +02:00
Marko Mäkelä
bc5cfe7769 Merge 10.2 into 10.3 2019-11-14 10:51:06 +02:00
Sujatha
caa79081c3 MDEV-20707: Missing memory barrier in parallel replication error handler in wait_for_prior_commit()
revision-id: 673e253724979fd9fe43a4a22bd7e1b2c3a5269e
Author: Kristian Nielsen

Fix missing memory barrier in wait_for_commit.

The function wait_for_commit::wait_for_prior_commit() has a fast path where it
checks without locks if wakeup_subsequent_commits() has already been called.
This check was missing a memory barrier. The waitee thread does two writes to
variables `waitee' and `wakeup_error', and if the waiting thread sees the
first write it _must_ also see the second or incorrect behavior will occur.
This requires memory barriers between both the writes (release semantics) and
the reads (acquire semantics) of those two variables.

Other accesses to these variables are done under lock or where only one thread
will be accessing them, and can be done without barriers (relaxed semantics).
2019-11-14 12:03:39 +05:30
Marko Mäkelä
c99470b366 Merge 10.4 into 10.5 2019-11-13 20:38:14 +02:00
Marko Mäkelä
49019dde65 MDEV-17138 follow-up: Optimize index page creation
btr_create(), btr_root_raise_and_insert(): Write a MLOG_MEMSET record
to set FIL_PAGE_PREV,FIL_PAGE_NEXT to FIL_NULL, instead of writing
two MLOG_4BYTES records.

For ROW_FORMAT=COMPRESSED pages, we will not use MLOG_MEMSET
because we want the crash-downgrade to earlier 10.4 releases to succeed.

mlog_parse_nbytes(): Relax the too strict assertion. There is no problem
with MLOG_MEMSET records that affect the uncompressed header of
ROW_FORMAT=COMPRESSED index pages.
2019-11-13 18:35:04 +02:00
Sergei Petrunia
d4edb0510e MDEV-20646: 10.3.18 is slower than 10.3.17
Fix incorrect change introduced in the fix for MDEV-20109.

The patch tried to compute a more precise estimate for the record_count
value in SJ-Materialization-Scan strategy (in
Sj_materialization_picker::check_qep).  However the new formula is worse
as it produces extremely optimistic results in common cases where
SJ-Materialization-Scan should be used)

The old formula produces pessimistic results in cases when Sj-Materialization-
Scan is unlikely to be a good choice anyway. So, the old behavior is better.
2019-11-13 18:53:59 +03:00
Eugene Kosov
98694ab0cb MDEV-20949 Stop issuing 'row size' error on DML
Move row size check to early CREATE/ALTER TABLE phase. Stop checking
on table open.

dict_index_add_to_cache(): remove parameter 'strict', stop checking row size

dict_index_t::record_size_info_t: this is a result of row size check operation

create_table_info_t::row_size_is_acceptable(): performs row size check.
Issues error or warning. Writes first overflow field to InnoDB log.

create_table_info_t::create_table(): add row size check

dict_index_t::record_size_info(): this is a refactored version
of dict_index_t::rec_potentially_too_big(). New version doesn't change global
state of a program but return all interesting info. And it's callers who
decide how to handle row size overflow.

dict_index_t::rec_potentially_too_big(): removed
2019-11-13 22:00:55 +07:00
Alexander Barkov
29b11cffb5 MDEV-21043 Collect different bison %type declarations into a single chunk 2019-11-13 17:47:18 +04:00
Marko Mäkelä
2b7aa60b7e Use constexpr for constants on data pages 2019-11-13 14:34:52 +02:00
Marko Mäkelä
3b573c0783 Clean up mtr_t::commit() further
memo_block_unfix(), memo_latch_release(): Merge to ReleaseLatches.

memo_slot_release(), ReleaseAll: Clean up the formatting.
2019-11-13 09:51:28 +02:00
Marko Mäkelä
abd45cdc38 MDEV-20934: Correct a debug assertion
A search with PAGE_CUR_GE may land on the supremum record on
a leaf page that is not the rightmost leaf page.
This could occur when all keys on the current page are
smaller than the search key, and the smallest key on the
successor page is larger than the search key.

ibuf_delete_recs(): Correct the debug assertion accordingly.
2019-11-13 09:26:10 +02:00
Eugene Kosov
ae72205e31 cleanup: replace List_iterator(_fast) in handler0alter.cc
Basically, use more List<T>::iterator. This patch required adding two more
overloads to new iterator for convenience.
2019-11-13 01:23:46 +07:00
Kentoku SHIBA
83a0eaec08
MDEV-18987 bug in "load data local infile xxx replace into " (#1408)
It's just added regression tests.
2019-11-13 00:32:27 +09:00
Marko Mäkelä
1fe6e5a1a5 Merge 10.4 into 10.5 2019-11-12 17:21:37 +02:00
Yasuhiro Horimoto
f127fb9807 Fix a typo in mariadb-plugin-mroonga.prerm
Closes #1407
2019-11-12 07:00:15 -08:00
Marko Mäkelä
33cb10d4e9 Merge 10.3 into 10.4 2019-11-12 16:55:44 +02:00
Marko Mäkelä
5098d708a0 Merge 10.2 into 10.3 2019-11-12 16:42:58 +02:00
Marko Mäkelä
2570cb8b91 MDEV-12353 preparation: Clean up mtr_t
mtr_t::Impl, mtr_t::Command: Merge to mtr_t.

MTR_MAGIC_N: Remove.

MTR_STATE_COMMITTING: Remove. This state was only being set
internally during mtr_t::commit().

mtr_t::Command::m_locks_released: Remove (set-and-never-read member).

mtr_t::Command::m_start_lsn: Replaced with the return value of
finish_write() and a parameter to release_blocks().

mtr_t::Command::m_end_lsn: Removed as a duplicate of mtr_t::m_commit_lsn.

mtr_t::Command::prepare_write(): Replace a switch () with a
comparison against 0. Only 2 m_log_mode are allowed.
2019-11-12 15:46:57 +02:00
Alexander Barkov
e26d049197 MDEV-21023 Move LEX methods and related functions from sql_yacc.yy to sql_lex.cc 2019-11-12 16:52:44 +04:00
Marko Mäkelä
dc8380b65d MDEV-14602: Cleanup recv_dblwr_t::find_page()
Avoid creating std::vector, and use single instead of double traversal.
2019-11-12 14:41:24 +02:00
Marko Mäkelä
2350066e63 Merge 10.1 into 10.2 2019-11-12 14:36:37 +02:00
Sergei Petrunia
68ed3a81f2 MDEV-20854: ANALYZE for statements: not clear where the time is spent
Count the "gap" time between table accesses and display it as
r_other_time_ms in the "table" element.

* The advantage of this approach is that it doesn't add any new
  my_timer_cycles() calls.
* The disadvantage is that the definition of what is done during
  "other time" is not that clear: it includes checking the WHERE
  (for this table), constructing index lookup tuple (for the next table)
  writing to GROUP BY temporary table (as we dont account for that time
  separately [yet], etc)
2019-11-12 14:40:00 +03:00
Sujatha
7df07c7666 MDEV-20953: binlog_encryption.rpl_corruption failed in buildbot due to wrong error code
Problem:
========
CURRENT_TEST: binlog_encryption.rpl_corruption

mysqltest: In included file "./include/wait_for_slave_io_error.inc":
...
At line 72: Slave stopped with wrong error code
**** Slave stopped with wrong error code: 1743 (expected 1595,1913) ****

Analysis:
========
The test emulates the corruption at the various stages of replication for
example in binlog file, in network and in relay log etc. It verifies that all
corruption cases are handled through appropriate error messages.

The test cases which emulate network failure expect following errors.
--ER_SLAVE_RELAY_LOG_WRITE_FAILURE (1595)
--ER_NETWORK_READ_EVENT_CHECKSUM_FAILURE (1743)

Ideally test should expect error codes as 1595 and 1743.
But the test actually waits on incorrect error code 1595,1913

Fix:
===
Added appropriate error code for 'ER_NETWORK_READ_EVENT_CHECKSUM_FAILURE'.
Replaced 1913 with 1743.
2019-11-12 16:31:08 +05:30
Eugene Kosov
e5f99a0c0c MDEV-20297 Support C++11 range-based for loop for List<T>
New iterator has the fastest possible implementation: just moves one pointer.
It's faster that List_iterator and List_iterator_fast: both do more on increment.

Overall patch brings:
1) work compile times
2) possibly(!) worse debug build performance
3) definitely better optimized build performance
4) ability to write less code
5) ability to write less bug-prone code
2019-11-12 17:48:14 +07:00
Andrei Elkin
40e65e878e rpl_semi_sync_gtid_reconnect results merge 2019-11-11 21:12:14 +02:00
Andrei Elkin
d103c5a489 merge 10.2->10.3 with conflict resolutions 2019-11-11 16:28:21 +02:00
Andrei Elkin
26fd880d5e manual merge 10.1->10.2 2019-11-11 16:03:43 +02:00
Marko Mäkelä
0117d0e65a Merge 10.4 into 10.5 2019-11-11 15:21:58 +02:00
Marko Mäkelä
0308de94ee MDEV-17138 follow-up: Optimize fseg_create()
fseg_create(): Initialize FSEG_FRAG_ARRY by a single MLOG_MEMSET record.

flst_zero_addr(), flst_init(): Optimize away redundant writes.

fseg_free_page_low(): Write FIL_NULL by MLOG_MEMSET.
2019-11-11 15:13:23 +02:00
Marko Mäkelä
3da895a736 Merge 10.3 into 10.4 2019-11-11 15:03:46 +02:00
Marko Mäkelä
4fcfdb60e7 Merge 10.2 into 10.3 2019-11-11 14:56:51 +02:00
Marko Mäkelä
142442d571 MDEV-21024: Cleanup XDES_CLEAN_BIT
The XDES_CLEAN_BIT is always set for every element of
the page allocation bitmap in the extent descriptor pages.
Do not bother touching it, to avoid redundant writes.
2019-11-11 14:18:50 +02:00
Marko Mäkelä
878bc854d9 MDEV-21024: Clean up dict_hdr_create()
The DICT_HDR_MAX_SPACE_ID was already zero-initialized at page allocation.
2019-11-11 14:15:04 +02:00
Marko Mäkelä
33f74e8fcf MDEV-21024: Clean up IMPORT TABLESPACE
page_rec_write_field(): Remove.

dict_create_index_tree_step(): If the SYS_INDEXES.PAGE does not change,
do not update it in the data dictionary. Typically, all index page numbers
would be unchanged before and after IMPORT TABLESPACE, except if some
secondary indexes were created after loading some data.

btr_root_fseg_adjust_on_import(): Remove the redundant mtr_t* parameter.
Redo logging is disabled during the page adjustments that IMPORT TABLESPACE
is performing.
2019-11-11 14:14:26 +02:00
Marko Mäkelä
dfdd96214b MDEV-21024: Clean up btr_root_raise_and_insert()
The root page must never have any siblings, so it is unnecessary
to clear those fields.
2019-11-11 14:14:26 +02:00
Marko Mäkelä
fc2ca2be4e MDEV-21024: Clean up page allocation
fsp_alloc_seg_inode_page(): Ever since
commit 3926673ce7
all newly allocated pages are zero-initialized.
Assert that this is the case for the FSEG_ID fields.
2019-11-11 14:14:26 +02:00
Marko Mäkelä
98e1d603bf MDEV-21024: Optimize writing BTR_EXTERN_LEN
btr_store_big_rec_extern_fields(): Remove the redundant initialization
of the most significant 32 bits of BTR_EXTERN_LEN. InnoDB never supported
BLOBs that are longer than 4GiB. In fact, dtuple_convert_big_rec()
would write emit an error message if a clustered index record tuple would
exceed 1,000,000,000 bytes in length.

The BTR_EXTERN_LEN in the BLOB pointers in clustered index leaf page
records is zero-initialized at least since
commit 41bb3537ba
2019-11-11 14:14:26 +02:00
Marko Mäkelä
3621df70ca MDEV-21024: Clean up rtr_adjust_upper_level()
Remove the unnecessary retrieval and null-modifications of
the preceding page.
2019-11-11 14:14:20 +02:00
Marko Mäkelä
29d67d051a Cleanup btr_page_get_prev(), btr_page_get_next()
Remove the redundant parameter mtr_t*.

Make use of page_has_prev(), page_has_next() whenever possible.
2019-11-11 13:36:21 +02:00
Marko Mäkelä
1d2458f813 MDEV-21024: Clean up rtr_adjust_upper_level()
Remove the unnecessary retrieval and null-modifications of
the preceding page.
2019-11-11 13:26:19 +02:00
Andrei Elkin
13db50fc03 MDEV-19376 Repl_semi_sync_master::commit_trx assertion failure: ... || !m_active_tranxs->is_tranx_end_pos(trx_wait_binlog_name, trx_wait_binlog_pos)
The assert indicates that the current transaction got caught uncleaned from
the semisync master's cache when it is signaled to proceed upon its
ack receive.

The reason of missed cleanup turns out to be a flaw in the gtid
connect mode.
A submitted by connecting slave value of its last received event's
binlog file *name* was adopted into
{{Repl_semi_sync_master::m_reply_file_name}} as a part of semisync
initialization.

Notice that the initialization still refines the position part of the
submitted last received event's binlog coordinates.
The master side binlog filename:pos refinement is
specific to the gtid connect mode for purpose of computing the latest
binlog file to resume slave feeding from.
Effectively in the gtid connect mode the computed resumption filename:pos
may appear smaller in which case a new post-connect time committing
transaction may be logged with its filename:pos also less than the
submitted coordinates and that triggers the assert.

Fixed with making the semisync initialization to use the refined filename:pos.
It is guaranteed to be less than any new generated transaction's binlog:pos.
2019-11-10 16:16:37 +02:00
Daniel Bartholomew
74892bda3a
bump the VERSION 2019-11-08 09:53:06 -05:00