Commit graph

21769 commits

Author SHA1 Message Date
Marko Mäkelä
437da7bc54 MDEV-19534 Make innodb_checksum_algorithm=full_crc32 by default, and remove innodb_checksums 2019-05-22 08:53:28 +03:00
Sergey Vojtovich
ec926b0f40 Fixed RocksDB to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in rocksdb_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Removed transaction objects cleanup from rocksdb_done_func(). As we lock
plugin properly, there must be no transaction objects during RocksDB
shutdown.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
8c8d584f06 Fixed InnoDB to not use broken thd_ha_data()
Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5e1b3cc8dc Fixed Aria to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
ba59cc0f37 Fixed Sphinx to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in sphinx_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
00e533c78b Fixed Mroonga to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in mrn_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
5c18ba6c88 Fixed Spider to follow THD ha_data protocol
Do not reset THD ha_data in spider_close_connection(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Sergey Vojtovich
762d2b96fa Fixed FederatedX to follow THD ha_data protocol
Use thd_get_ha_data()/thd_set_ha_data() which protect against plugin
removal until it has THD ha_data.

Do not reset THD ha_data in ha_federatedx::disconnect(), cleaner approach
is to let ha_close_connection() do it.

Part of MDEV-19515 - Improve connect speed
2019-05-21 17:55:09 +04:00
Marko Mäkelä
c3ea52c92a MDEV-17958: Remove IS_BIG_ENDIAN
In 62d28f83b6 the CMake variable
IS_BIG_ENDIAN became orphaned. Remove it.
2019-05-14 12:35:42 +03:00
Monty
341c3379ae Removed obsolete file maria_rename.sh 2019-05-13 13:59:41 +03:00
Monty
60518a6b85 Make maria-autozerofill a bit more rebust
- Remove warning about aria_control_file
- Write to unique test file

Other things:
- Fixed typo in aria_chk --help
2019-05-13 13:59:41 +03:00
Monty
44b8b002f5 Disable 5733_tokudb as the result is not stable 2019-05-09 11:24:06 +03:00
Vladislav Vaintroub
60bd353bdf Fixes for atomic writes on Windows.
Windows does atomic writes, as long as they are aligned and multiple
of sector size. this is documented in MSDN.

Fix innodb.doublewrite test to always use doublewrite buffer,
(even if atomic writes are autodetected)
2019-05-06 11:32:17 +00:00
Vladislav Vaintroub
59a266a9f9 MDEV-17380: fix incorrect #ifdef 2019-05-06 11:31:51 +00:00
Vladislav Vaintroub
c477623f04 MDEV-19388 Improve SSD detection on Windows
Fallback to detecting if TRIM is enabled, if we cannot determine
seek penalty.
2019-05-06 10:15:46 +00:00
Marko Mäkelä
d3dcec5d65 Merge 10.3 into 10.4 2019-05-05 15:06:44 +03:00
Marko Mäkelä
b132b8895e Merge 10.3 into 10.4 2019-05-05 10:23:14 +03:00
Marko Mäkelä
b6f4cccd19 Merge 10.2 into 10.3 2019-05-03 20:14:09 +03:00
Marko Mäkelä
ce195987c3 MDEV-19385: Inconsistent definition of dtuple_get_nth_v_field()
The accessor dtuple_get_nth_v_field() was defined differently between
debug and release builds in MySQL 5.7.8 in
mysql/mysql-server@c47e1751b7
and a debug assertion to document or enforce the questionable assumption
tuple->v_fields == &tuple->fields[tuple->n_fields] was missing.

This was apparently no problem until MDEV-11369 introduced instant
ADD COLUMN to MariaDB Server 10.3. With that work present, in one
test case, trx_undo_report_insert_virtual() could in release builds
fetch the wrong value for a virtual column.

We replace many of the dtuple_t accessors with const-preserving
inline functions, and fix missing or misleadingly applied const
qualifiers accordingly.
2019-05-03 20:02:50 +03:00
Marko Mäkelä
3db94d2403 MDEV-19346: Remove dummy InnoDB log checkpoints
log_checkpoint(), log_make_checkpoint_at(): Remove the parameter
write_always. It seems that the primary purpose of this parameter
was to ensure in the function recv_reset_logs() that both checkpoint
header pages will be overwritten, when the function is called from
the never-enabled function recv_recovery_from_archive_start().

create_log_files(): Merge recv_reset_logs() to its only caller.

Debug instrumentation: Prefer to flush the redo log, instead of
triggering a redo log checkpoint.

page_header_set_field(): Disable a debug assertion that will
always fail due to MDEV-19344, now that we no longer initiate
a redo log checkpoint before an injected crash.

In recv_reset_logs() there used to be two calls to
log_make_checkpoint_at(). The apparent purpose of this was
to ensure that both InnoDB redo log checkpoint header pages
will be initialized or overwritten.
The second call was removed (without any explanation) in MySQL 5.6.3:
mysql/mysql-server@4ca37968da

In MySQL 5.6.8 WL#6494, starting with
mysql/mysql-server@00a0ba8ad9
the function recv_reset_logs() was not only invoked during
InnoDB data file initialization, but also during a regular
startup when the redo log is being resized.

mysql/mysql-server@45e9167983
in MySQL 5.7.2 removed the UNIV_LOG_ARCHIVE code, but still
did not remove the parameter write_always.
2019-05-03 20:02:11 +03:00
Marko Mäkelä
5182348316 Merge 10.2 into 10.3 2019-05-02 21:45:13 +03:00
Thirunarayanan Balathandayuthapani
ada1074bb1 MDEV-14398 innodb_encryption_rotate_key_age=0 causes innodb_encrypt_tables to be ignored
The statement

SET GLOBAL innodb_encryption_rotate_key_age=0;

would have the unwanted side effect that ENCRYPTION=DEFAULT tablespaces
would no longer be encrypted or decrypted according to the setting of
innodb_encrypt_tables.

We implement a trigger, so that whenever one of the following is executed:

SET GLOBAL innodb_encrypt_tables=OFF;
SET GLOBAL innodb_encrypt_tables=ON;
SET GLOBAL innodb_encrypt_tables=FORCE;

all wrong-state ENCRYPTION=DEFAULT tablespaces will be added to
fil_system_t::rotation_list, so that the encryption will be added
or removed.

Note: This will *NOT* happen automatically after a server restart.
Before reading the first page of a data file, InnoDB cannot know
the encryption status of the data file. The statement
SET GLOBAL innodb_encrypt_tables will have the side effect that
all not-yet-read InnoDB data files will be accessed in order to
determine the encryption status.

innodb_encrypt_tables_validate(): Stop disallowing
SET GLOBAL innodb_encrypt_tables when innodb_encryption_rotate_key_age=0.
This reverts part of commit 50eb40a2a8
that addressed MDEV-11738 and MDEV-11581.

fil_system_t::read_page0(): Trigger a call to fil_node_t::read_page0().
Refactored from fil_space_get_space().

fil_crypt_rotation_list_fill(): If innodb_encryption_rotate_key_age=0,
initialize fil_system->rotation_list. This is invoked both on
SET GLOBAL innodb_encrypt_tables and
on SET GLOBAL innodb_encryption_rotate_key_age=0.

fil_space_set_crypt_data(): Remove.

fil_parse_write_crypt_data(): Simplify the logic.

This is joint work with Marko Mäkelä.
2019-05-02 13:31:59 +03:00
Kentoku SHIBA
0cbc930616
MDEV-18992 Crash when using 'insert into on duplicate update'if session charset different from table charset (#1290) 2019-04-30 21:39:08 +09:00
Marko Mäkelä
447b8ba164 Merge 10.2 into 10.3 2019-04-29 17:54:10 +03:00
Marko Mäkelä
810f014ca7 MDEV-18429: Simpler implementation
This reverts commit 61f370a3c9
and implements a simpler fix that is straightforward to merge to 10.3.

lock_print_info: Renamed from PrintNotStarted. Dump the
entire contents of trx_sys->mysql_trx_list.

lock_print_info_rw_recovered: Like lock_print_info, but dump
only recovered transactions in trx_sys->rw_trx_list.

lock_print_info_all_transactions(): Dump both trx_sys->mysql_trx_list
and trx_sys->rw_trx_list.

TrxLockIterator, TrxListIterator, lock_rec_fetch_page(): Remove.
This is a partial backport of the 10.3
commit a447980ff3
which removed the race-condition-prone ability of the InnoDB monitor
to read relevant pages into the buffer pool for some record locks.
2019-04-29 17:49:55 +03:00
Marko Mäkelä
092602ac9b MDEV-14130 InnoDB messages should not refer to the MySQL 5.7 manual
In InnoDB error messages, replace the hyperlink URLs to point
to the MariaDB knowledge base.
2019-04-29 15:11:06 +03:00
Marko Mäkelä
5fb4c0a8a8 Clean up ut_list
ut_list_validate(), ut_list_map(): Add variants with const Functor&
so that these functions can be called with an rvalue.

Remove wrapper macros, and add #ifdef UNIV_DEBUG around debug-only code.
2019-04-29 15:11:06 +03:00
Eugene Kosov
61f370a3c9 MDEV-18429 Consistent non-locking reads do not appear in TRANSACTIONS section of SHOW ENGINE INNODB STATUS
lock_print_info_all_transactions(): print transactions which are started
but not in RW or RO lists.

print_not_started(): Replaces PrintNotStarted. Collect the skipped
transactions.
2019-04-29 15:11:06 +03:00
Marko Mäkelä
1b577e4d8b MDEV-19356 Assertion 'space->free_limit == 0 || space->free_limit == free_limit'
fil_node_t::read_page0(): Do not replace up-to-date metadata with a
possibly old version of page 0 that is being reread from the file.
A more up-to-date page 0 could still exists in the buffer pool,
waiting to be written back to the file.
2019-04-29 11:43:22 +03:00
Marko Mäkelä
e10b3fa97a MDEV-19231: Correct an assertion
BtrBulk::finish(): On error, the B-tree can be corrupted. Only upon
successful completion, it makes sense to validate the index.
2019-04-29 10:04:54 +03:00
Marko Mäkelä
4c995eb168 Correct the result for a few tests
These were broken in commit b5615eff0d
2019-04-29 09:32:43 +03:00
Marko Mäkelä
2f1f02817e After-merge fix for rocksdb
Adjust a test for the merge commit e6bdf77e4b
2019-04-29 09:27:32 +03:00
Marko Mäkelä
4d59f45260 Merge 10.2 into 10.3 2019-04-27 20:41:31 +03:00
Marko Mäkelä
1a5ba2a4be MDEV-19342 Merge new release of InnoDB 5.7.26 to 10.2 2019-04-26 18:19:50 +03:00
Marko Mäkelä
4e9f8c9cc4 Remove roll_node_t::partial
The field roll_node_t::partial holds if and only if
savept has been set. Make savept a pointer.

trx_rollback_start(): Use the semantic type undo_no_t for roll_limit.
2019-04-26 17:40:20 +03:00
Aditya A
f3a9f12bc3 Bug #29021730 CRASHING INNOBASE_COL_CHECK_FK WITH FOREIGN KEYS
PROBLEM
-------
Function innodb_base_col_setup_for_stored() was skipping to store
the base column information for a generated column if the base column
was a "STORED" generated column. This later causes a crash in function
innoabse_col_check_fk() where it says that a generated columns depends
upon two base columns ,but there is information on only one of them.
There was a explicit check barring the stored columns being stored,
which is wrong because the documentation says that a generated stored
column can be a part of a generated column.

FIX
----
Store the information of base column if it is a stored generated column.

#RB21247
Reviewed by: Debarun Banerjee <debarun.banerjee@oracle.com>
2019-04-26 17:40:20 +03:00
Marko Mäkelä
793bd3ee13 lock_rec_convert_impl_to_expl_for_trx(): Remove unused parameter 2019-04-26 17:40:20 +03:00
Sachin Agarwal
06ec56f579 Bug #27850600 INNODB ASYNC IO ERROR HANDLING IN IO_EVENT
Problem:
io_getevents() - read asynchronous I/O events from the completion
queue. For each IO event, the res field in io_event tells whether IO
event is succeeded or not. To see if the IO actually succeeded we
always need to check event.res (negative=error,
positive=bytesread/written).
LinuxAIOHandler::collect() doesn't check event.res value for each event.
which leads to incorrect value in n_bytes for IO context (or IO Slot).

Fix:
Added a check for event.res negative value.

RB: 20871
Reviewed by : annamalai.gurusami@oracle.com
2019-04-26 17:40:20 +03:00
Marko Mäkelä
1c4d1f3d05 innobase_col_check_fk(): Remove copying 2019-04-26 17:40:19 +03:00
Varun Gupta
d1a43973ef Adjusted result for tokudb_bugs.db756_card_part_hash_2_pick 2019-04-26 12:50:26 +05:30
Varun Gupta
87472974cd Results updated for tokudb tests 2019-04-25 22:05:54 +05:30
Kentoku SHIBA
6599cd985e
MDEV-18988 Wrong result when query with group by x order by y limit n (#1286) 2019-04-26 00:51:28 +09:00
Kentoku SHIBA
82d0698962
MDEV-18995 Some bugs in direct join (#1285)
Add const_table support to the direct join feature and condition pushdown feature
2019-04-25 23:28:04 +09:00
Marko Mäkelä
b2dbc781c7 Implement --debug=d,ib_log_checkpoint_avoid
Normally, the InnoDB master thread executes InnoDB log checkpoints
so frequently that bugs in crash recovery or redo logging can be
hard to reproduce. This is because crash recovery would start replaying
the log only from the latest checkpoint. Because the InnoDB redo log
format only allows saving information for at most 2 latest checkpoints,
and because the log files are written in a circular fashion, it would
be challenging to implement a debug option that would start the redo
log apply from the very start of the redo log file.
2019-04-25 17:26:23 +03:00
Eugene Kosov
6c5c1f0b2f MDEV-19231 make DB_SUCCESS equal to 0
It's a micro optimization. On most platforms CPUs has instructions to
compare with 0 fast. DB_SUCCESS is the most popular outcome of functions
and this patch optimized code like (err == DB_SUCCESS)

BtrBulk::finish(): bogus assertion fixed

fil_node_t::read_page0(): corrected usage of os_file_read()

que_eval_sql(): bugus assertion removed. Apparently it checked that
the field was assigned after having been zero-initialized at
object creation.

It turns out that the return type of os_file_read_func() was changed
in mysql/mysql-server@98909cefbc (MySQL 5.7)
from ibool to dberr_t. The reviewer (if there was any) failed to
point out that because of future merges, it could be a bad idea to
change the return type of a function without changing the function name.

This change was applied to MariaDB 10.2.2 in
commit 2e814d4702 but the
MariaDB-specific code was not fully adjusted accordingly,
e.g. in fil_node_open_file(). Essentially, code like
!os_file_read(...) became dead code in MariaDB and later
in Mariabackup 10.2, and we could be dealing with an uninitialized
buffer after a failed page read.
2019-04-25 16:29:55 +03:00
Marko Mäkelä
e6bdf77e4b Merge 10.3 into 10.4
In is_eits_usable(), we disable an assertion that fails due to
MDEV-19334.
2019-04-25 16:05:20 +03:00
Sergey Vojtovich
210855ce5d Move XID_STATE::xid to XID_cache_element
Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)
2019-04-25 15:06:40 +04:00
Sergey Vojtovich
a168cfb396 Move XID_state::xa_state handing inside xa.cc
Let xid_cache_insert()/xid_cache_delete() handle xa_state.

Let session tracker use is_explicit_XA() rather than xa_state != XA_NOTR.

Fixed open_tables() to refuse data access in XA_ROLLBACK_ONLY state.

Removed dead code from THD::cleanup(). It was supposed to be a reminder,
but it got messed up over time.

spider_internal_start_trx() is called either with XA_NOTR or XA_ACTIVE,
which is guarded by server callers. Thus is_explicit_XA() is acceptable
replacement for XA_ACTIVE check (which was likely wrong anyway).

Setting xa_state to XA_PREPARED in spider_internal_xa_prepare() isn't
meaningful, as this value is never accessed later. It can't be accessed
by current thread and it can't be recovered either. It can only be
accessed by spider internally, which never happens.

Make spider_xa_lock()/spider_xa_unlock() static.

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)
2019-04-25 15:06:40 +04:00
Marko Mäkelä
acf6f92aa9 Merge 10.2 into 10.3 2019-04-25 09:05:52 +03:00
Marko Mäkelä
bc145193c1 Merge 10.1 into 10.2 2019-04-25 09:04:09 +03:00