Commit graph

23332 commits

Author SHA1 Message Date
Daniel Black
ba2061da52 MDEV-21595: innodb offset_t rename to rec_offs
thanks to:

perl -i -pe 's/\boffset_t\b/rec_offs/g' $(git grep -lw offset_t storage/innobase)
2020-04-29 12:02:47 +03:00
Marko Mäkelä
cfbbf5424b MDEV-7962: Follow-up fix for 10.4
Replace wsrep_on() with trx_t::is_wsrep() where possible.

Also, rename some functions to member functions and
remove unused DBUG_EXECUTE_IF instrumentation:

trx_t::commit(): Renamed from trx_commit().

trx_t::commit_low(): Renamed from trx_commit_low().

trx_t::commit_in_memory(): Renamed from trx_commit_in_memory().
2020-04-29 11:50:03 +03:00
Marko Mäkelä
0632b8034b Merge 10.3 into 10.4 2020-04-29 09:05:15 +03:00
Marko Mäkelä
1fbdcada73 Merge 10.2 into 10.3 2020-04-28 22:29:13 +03:00
Marko Mäkelä
dd5c307cb0 MDEV-22394 Merge new release of InnoDB 5.7.30 to 10.2 2020-04-28 19:40:32 +03:00
Marko Mäkelä
2644e52fdb MDEV-22384 Wrong estimate of affected BLOB columns in update of PRIMARY KEY
During the UPDATE of PRIMARY KEY columns, we may miscalculate the
size of the clustered index record.

row_upd_clust_rec_by_insert(): Pass the total number of off-page columns,
which may include such columns that were inherited from the record
and not created as part of the UPDATE operation.

This is based on
mysql/mysql-server@490c45e8c8
which is a follow-up to
mysql/mysql-server@1fa475b85d
which we filed and fixed as MDEV-21511.

No test case was provided by Oracle.
2020-04-28 19:39:51 +03:00
Marko Mäkelä
547cb280b8 Merge 10.1 into 10.2 2020-04-28 19:39:40 +03:00
Marko Mäkelä
fae70d6b1c Cleanup: Declare rtr_get_father_node() statically 2020-04-28 19:37:07 +03:00
Thirunarayanan Balathandayuthapani
5193c1b542 MDEV-22369 Assertion `err == DB_SUCCESS' failed. in btr_block_get_func during crash recovery
- buf_page_get_gen() fails to set error value when it initializes the
page with redo logs.
2020-04-28 19:23:51 +05:30
Marko Mäkelä
713e427b2e InnoDB 5.6.48 2020-04-28 16:20:19 +03:00
Marko Mäkelä
7041807476 MDEV-22393 Corruption for SET GLOBAL innodb_ string variables
Several MYSQL_SYSVAR_STR parameters that employ both a validate
function callback fail to copy the string for saving the
validated value. The affected variables include the following:

innodb_ft_aux_table
innodb_ft_server_stopword_table
innodb_ft_user_stopword_table
innodb_buffer_pool_filename

The test case is an enhanced version of
mysql/mysql-server@0b0c30641f
and the code changes are inspired by their fixes.

We are also importing and adjusting the test innodb_fts.stopword
to get coverage for the variable innodb_ft_user_stopword_table.

buf_dump(), buf_load(): Protect srv_buf_dump_filename with
LOCK_global_system_variables.

fts_load_user_stopword(): Minor cleanup

fts_load_stopword(): Remove the parameter global_stopword_table.

innobase_fts_load_stopword(): Protect innodb_server_stopword_table
against concurrent SET GLOBAL.
2020-04-28 16:09:07 +03:00
Eugene Kosov
0cd2b4c249 MDEV-22177 more fsync() -> fdatasync() in InnoDB
Replace all fsync() with fdatasync() when possible (e.g. On Linux)

InnoDB doesn't care about file timestamps. So, to achieve a better
performance it makes sense to use fdatasync() everywhere.

file_io::flush(): renamed from flush_data_only()

os_file_flush_data(): removed

os_file_sync_posix(): renamed from os_file_fsync_posix(). Now it uses
fdatasync() when it's available.
2020-04-28 12:15:00 +03:00
Marko Mäkelä
cce1b6e245 MDEV-22392 Race condition on SET GLOBAL innodb_buffer_pool_evict='uncompressed'
innodb_buffer_pool_evict_uncompressed(): Restart the loop when
prev_block might not enjoy mutex protection.

This is based on
mysql/mysql-server@eccaecac07
2020-04-28 11:46:29 +03:00
Marko Mäkelä
cf64d27bad Remove a duplicated copyright message 2020-04-28 11:40:04 +03:00
Thirunarayanan Balathandayuthapani
c755974775 MDEV-19611 INPLACE ALTER does not fail on bad implicit default value
- Inplace alter shouldn't set default date column as '0000-00-00' when
table is not empty. So mysql_inplace_alter_table() copied
alter_ctx.error_if_not_empty to a new field of Alter_inplace_info.
In ha_innobase::check_if_supported_inplace_alter() should check the
error_if_not_empty flag and return INPLACE_NOT_SUPPORTED if the table
is not empty
2020-04-28 11:59:32 +05:30
Marko Mäkelä
b63446984c Merge 10.3 into 10.4 2020-04-27 17:38:17 +03:00
Marko Mäkelä
3568fad5c9 Merge 10.2 into 10.3 2020-04-27 16:06:26 +03:00
Marko Mäkelä
581df0df89 MDEV-7962: Follow-up fix for 10.2
dict_stats_update_if_needed(): Replace the parameter THD*
with const trx_t& so that trx_t::is_wsrep() can be invoked
instead of the more expensive wsrep_on().

Replace also other occurrences of wsrep_on() with trx_t::is_wsrep().
2020-04-27 15:32:28 +03:00
Marko Mäkelä
2e12d471ea Merge 10.2 into 10.3 2020-04-27 14:24:41 +03:00
Marko Mäkelä
c06845d6f0 Merge 10.1 into 10.2 2020-04-27 13:28:13 +03:00
Marko Mäkelä
d956175d0d XtraDB 5.6.47-87.0
The only change is a change of the version number.
As noted in commit 02af6278fb
there were no changes to InnoDB between MySQL 5.6.46 and 5.6.47
either.
2020-04-27 11:39:46 +03:00
Marko Mäkelä
edbdfc2f99 MDEV-7962 wsrep_on() takes 0.14% in OLTP RO
The function wsrep_on() was being called rather frequently
in InnoDB and XtraDB. Let us cache it in trx_t and invoke
trx_t::is_wsrep() instead.

innobase_trx_init(): Cache trx->wsrep = wsrep_on(thd).

ha_innobase::write_row(): Replace many repeated calls to current_thd,
and test the cheapest condition first.
2020-04-27 11:18:11 +03:00
Marko Mäkelä
758fbec6e3 Fix clang 10 warnings
_ma_fetch_keypage(): Correct an assertion that used to always hold.
Thanks to clang -Wint-in-bool-context for flagging this.

double_to_datetime_with_warn(): Suppress -Wimplicit-int-float-conversion
by adding a cast. LONGLONG_MAX converted to double will actually be
LONGLONG_MAX+1.
2020-04-27 09:40:34 +03:00
Marko Mäkelä
6a3fc1101c Aria: Avoid unused variables in embedded server 2020-04-27 09:40:10 +03:00
Monty
81511b4776 Fixed shutdown crash in Aria that affects debug binaries
MDEV-18286 Assertion `pagecache->cnt_for_resize_op == 0' failed in
check_pagecache_is_cleaned_up on server shutdown

The reason for the crash is that the counter-of-pinned-pages in the
Aria pagecache goes wrong.
This only affects debug builds, as in these we do an assert on shutdown
if the counter-of-pinned-pages is not 0 (some page was left pinned).

The bug was that in 2 places in the page cache, when not succeeding to
pin a page and a retry was made, the counter-of-pinned-pages counter was
not properly adjusted.

In the given test case, BLOCK_COMMIT flushed all Aria files. If a block was flushed at the same time the insert tried to access it, the insert would retry to get the block and that would cause the counter to go wrong.
2020-04-26 15:26:48 +03:00
Marko Mäkelä
fbe2712705 Merge 10.4 into 10.5
The functional changes of commit 5836191c8f
(MDEV-21168) are omitted due to MDEV-742 having addressed the issue.
2020-04-25 21:57:52 +03:00
Eugene Kosov
62903434eb fix s390x warning caused by gcc-5 bug 2020-04-25 21:46:31 +03:00
Marko Mäkelä
edd38b50f6 MDEV-7962 wsrep_on() takes 0.14% in OLTP RO
The reason why we have wsrep_on() at all is that the macro WSREP(thd)
depends on the definition of THD, and that is intentionally an opaque
data type for InnoDB. So, we cannot avoid invoking wsrep_on(), but
we can evaluate the less expensive conditions thd && WSREP_ON before
calling the function.

Global_read_lock: Use WSREP_NNULL(thd) instead of wsrep_on(thd)
because we not only know the definition of THD but also that
the pointer is not null.

wsrep_open(): Use WSREP(thd) instead of wsrep_on(thd).

InnoDB: Replace thd && wsrep_on(thd) with wsrep_on(thd), now that
the condition has been merged to the definition of the macro
wsrep_on().
2020-04-24 16:01:10 +03:00
Marko Mäkelä
2c39f69d34 MDEV-22203: WSREP_ON is unnecessarily expensive WITH_WSREP=OFF
If the server is compiled WITH_WSREP=OFF, we should avoid evaluating
conditions on a global variable that is constant.

WSREP_ON_: Renamed from WSREP_ON. Defined only WITH_WSREP=ON.

WSREP_ON: Defined as unlikely(WSREP_ON_).

wsrep_on(): Defined as WSREP_ON && wsrep_service->wsrep_on_func().

The reason why we have wsrep_on() at all is that the macro WSREP(thd)
depends on the definition of THD, and that is intentionally an opaque
data type for InnoDB. So, we cannot avoid invoking wsrep_on(), but
we can evaluate the less expensive condition WSREP_ON before calling
the function.
2020-04-24 15:25:39 +03:00
Eugene Kosov
2b2dcf34f7 fix buggy gcc warning on ARM
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89357

In short, alignas of big alignments doesn't work for gcc on ARM.
The bug is fixed in gcc-9.3 and won't be fixed in older branches.
F.ex gcc-7 is not supported already.
2020-04-24 12:09:49 +03:00
Marko Mäkelä
da7564edcf Cleanup: Make row_upd_store_row() static 2020-04-24 10:46:33 +03:00
Marko Mäkelä
57ec41d6ea Cleanup: Remove a constant parameter
row_vers_vc_matches_cluster(): Remove the parameter in_purge,
which was always passed as in_purge=true.

This parameter became constant in
mysql/mysql-server@1dec14d346
and it always was constant in MariaDB starting from the
introduction of the function in
commit 2e814d4702 (MariaDB 10.2.2).
2020-04-24 10:42:08 +03:00
Marko Mäkelä
9f19dbe0c3 MDEV-22358 Assertion srv_undo_sources || ... in row_prebuilt_free()
row_prebuilt_free(): Do not attempt to drop orphan indexes
that might have been left behind by a failed ADD UNIQUE INDEX.
This avoids the execution of unwanted transactions during shutdown.
2020-04-23 19:33:55 +03:00
Monty
9398c3dfa5 Fixed memory leak if Aria didn't start
This is not crtitical as if Aria doesn't start, the server will exit at once.
2020-04-23 14:24:52 +03:00
Marko Mäkelä
88cf6f1c7f Merge 10.3 into 10.4 2020-04-22 18:18:51 +03:00
Kentoku SHIBA
498c5e2be1 MDEV-21884 MariaDB with Spider crashes on a query
Fix a test result.
2020-04-23 00:03:54 +09:00
Marko Mäkelä
455cf6196c Merge 10.2 into 10.3 2020-04-22 14:45:55 +03:00
Marko Mäkelä
e52a36d37b Merge 10.1 into 10.2 2020-04-22 13:50:53 +03:00
Marko Mäkelä
30c9833751 MDEV-22332: Assertion mtr_started == mtr.is_active() failed
Commit 5defdc382b
(which aimed to reduce sizeof(mtr_t) for non-debug builds)
replaced the ternary mtr_t::status with two debug-only bool
data members m_start, m_commit and inadvertently made the
(now debug-only) predicate mtr_t::is_active() wrongly hold
after mtr_t::commit().

mtr_t::is_active(): Evaluate both m_start and m_commit,
to be compatible with the old definition.

row_merge_read_clustered_index(): Correct a debug assertion.
2020-04-22 12:36:11 +03:00
Kentoku SHIBA
1b81e96593 MDEV-21884 MariaDB with Spider crashes on a query
Fix a test result.
2020-04-22 04:44:28 +09:00
Marko Mäkelä
4db43972e1 MDEV-18115: Remove fil_type_is_data()
When commit 562c037b48
removed FIL_TYPE_LOG, the function fil_type_is_data()
became redundant, that is, always returning true for
a valid value of fil_space_t::purpose. Remove it.
2020-04-21 15:04:21 +03:00
Monty
8d74d30dde Fixed compiler warning 2020-04-19 17:33:51 +03:00
Monty
f40ca33bbc Make all #sql temporary table names uniform
The reason for this is to make all temporary file names similar and
also to be able to figure out from where a #sql-xxx name orginates.

New format is for most cases:
'#sql-name-current_pid-thread_id[-increment]'
Where name is one of subselect, alter, exchange, temptable or backup

The exceptions are:
ALTER PARTITION shadow files:
'#sql-shadow-thread_id-'original_table_name'

Names used with temp pool:
'#sql-name-current_pid-pool_number'
2020-04-19 17:33:51 +03:00
Monty
eca5c2c67f Added support for more functions when using partitioned S3 tables
MDEV-22088 S3 partitioning support

All ALTER PARTITION commands should now work on S3 tables except

REBUILD PARTITION
TRUNCATE PARTITION
REORGANIZE PARTITION

In addition, PARTIONED S3 TABLES can also be replicated.
This is achived by storing the partition tables .frm and .par file on S3
for partitioned shared (S3) tables.

The discovery methods are enchanced by allowing engines that supports
discovery to also support of the partitioned tables .frm and .par file

Things in more detail

- The .frm and .par files of partitioned tables are stored in S3 and kept
  in sync.
- Added hton callback create_partitioning_metadata to inform handler
  that metadata for a partitoned file has changed
- Added back handler::discover_check_version() to be able to check if
  a table's or a part table's definition has changed.
- Added handler::check_if_updates_are_ignored(). Needed for partitioning.
- Renamed rebind() -> rebind_psi(), as it was before.
- Changed CHF_xxx hadnler flags to an enum
- Changed some checks from using table->file->ht to use
  table->file->partition_ht() to get discovery to work with partitioning.
- If TABLE_SHARE::init_from_binary_frm_image() fails, ensure that we
  don't leave any .frm or .par files around.
- Fixed that writefrm() doesn't leave unusable .frm files around
- Appended extension to path for writefrm() to be able to reuse to function
  for creating .par files.
- Added DBUG_PUSH("") to a a few functions that caused a lot of not
  critical tracing.
2020-04-19 17:33:51 +03:00
Monty
a6d32976ae Fixed memory leak with fulltext indexes
MDEV-22275 Assertion `global_status_var.global_memory_used == 0'
failed, bytes lost, or LeakSanitizer: detected memory leaks
after using temporary table with fulltext key

This affected MyISAM and Aria temporary tables
2020-04-18 11:51:42 +03:00
Marko Mäkelä
ad4b70562b Fix GCC 10 -Woverflow
maria_page_crc_check_index(): Do not attempt to convert
HA_ERR_WRONG_CRC (176) to my_bool (char).
On platforms where char is signed, the 176 will be converted to -80.

It turns out that the callers only care whether the result is zero.
Let us return 1 in this case, like we do in all other error cases.
2020-04-17 12:02:22 +03:00
Kentoku SHIBA
181f17c3cd MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2020-04-17 11:52:42 +09:00
Kentoku SHIBA
68ceb4b460 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2020-04-17 11:51:14 +09:00
Kentoku SHIBA
b4dd996dc1 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 10:32:57 +09:00
Kentoku SHIBA
00db9c6b40 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 10:31:17 +09:00