Commit graph

19932 commits

Author SHA1 Message Date
Marko Mäkelä
0f90728bc0 MDEV-16809 Allow full redo logging for ALTER TABLE
Introduce the configuration option innodb_log_optimize_ddl
for controlling whether native index creation or table-rebuild
in InnoDB should keep optimizing the redo log
(and writing MLOG_INDEX_LOAD records to ensure that
concurrent backup would fail).

By default, we have innodb_log_optimize_ddl=ON, that is,
the default behaviour that was introduced in MariaDB 10.2.2
(with the merge of InnoDB from MySQL 5.7) will be unchanged.

BtrBulk::m_trx: Replaces m_trx_id. We must be able to check for
KILL QUERY even if !m_flush_observer (innodb_log_optimize_ddl=OFF).

page_cur_insert_rec_write_log(): Declare globally, so that this
can be called from PageBulk::insert().

row_merge_insert_index_tuples(): Remove the unused parameter trx_id.

row_merge_build_indexes(): Enable or disable redo logging based on
the innodb_log_optimize_ddl parameter.

PageBulk::init(), PageBulk::insert(), PageBulk::finish(): Write
redo log records if needed. For ROW_FORMAT=COMPRESSED, redo log
will be written in PageBulk::compress() unless we called
m_mtr.set_log_mode(MTR_LOG_NO_REDO).
2018-07-26 08:44:42 +03:00
Marko Mäkelä
32eb5823e4 Remove pointer indirection for BtrBulk::m_page_bulks 2018-07-26 08:44:41 +03:00
Marko Mäkelä
4a456eac2a Remove unused BtrBulk::m_heap 2018-07-26 08:44:41 +03:00
Marko Mäkelä
a3b22147ca Remove pointer indirection for PageBulk::m_mtr 2018-07-26 08:44:41 +03:00
Marko Mäkelä
172199b88c PageBulk: Remove dead code
Native ALTER TABLE is never invoked on temporary tables.
2018-07-26 08:44:41 +03:00
Marko Mäkelä
7f54894329 Remove mtr_set_flush_observer() 2018-07-26 08:44:41 +03:00
Thirunarayanan Balathandayuthapani
de85355436 MDEV-16713 Hangs server with repeating log entry
At most one transaction can be active at a time for temporary
tables. There is no need to check previous version of record for the
temporary tables.
2018-07-25 13:56:39 +05:30
Jacob Mathew
45ab00f097 MDEV-15786: ERROR 1062 (23000) at line 365: Duplicate entry 'spider' for key 'PRIMARY'
The problem occurs on Ubuntu where a Spider package is installed on the system
separately from the MariaDB package.  MariaDB and Spider upgrades leave the
Spider plugin improperly installed.  Spider is present in the mysql.plugin
table but is not present in information_schema.

The problem has been corrected in Spider's installation script.  Logic has
been added to check for Spider entries in both information_schema and
mysql.plugin.  If Spider is present in mysql.plugin but is not present in
information_schema, then Spider is first removed from mysql.plugin.  The
subsequent plugin install of Spider will insert entries in both mysql.plugin
and information_schema.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 0897d81 on branch bb-10.3-MDEV-15786
2018-07-23 14:14:23 -07:00
Marko Mäkelä
9d1f3bf2e9 row_purge_poss_sec(): Add debug instrumentation
This helped fix MDEV-16779.
2018-07-23 18:34:06 +03:00
Marko Mäkelä
c5ba13dda0 MDEV-15855 cleanup: Privatize purge_vcol_info_t
Declare all fields of purge_vcol_info_t private, and add
accessor functions.
2018-07-23 18:31:42 +03:00
Marko Mäkelä
a7a0c533c2 Follow-up to MDEV-15855: Remove bogus debug assertions
During a table-rebuilding operation, the function table_name_parse()
can encounter a table name that starts with #sql. Here is an example
of a failure:

CURRENT_TEST: gcol.innodb_virtual_basic
mysqltest: At line 1204: query 'alter table t drop column d ' failed:
2013: Lost connection to MySQL server during query

Let us just remove these bogus debug assertions.

If the final renaming phase during ALTER TABLE never fails, it
should not do any harm to skip the purge. If it does fail, then
we might end up 'leaking' some delete-marked records in the
indexes on virtual columns of the original table, and these
garbage records would keep consuming space until the indexes are
dropped or the table is successfully rebuilt.
2018-07-23 17:50:56 +03:00
Thirunarayanan Balathandayuthapani
730f6c912c MDEV-16779 Assertion !rw_lock_own failed upon purge
This is a regression caused by the fix of MDEV-15855.

purge_vcol_info_t::set_used(): Add a missing condition.

row_purge_poss_sec(): Invoke set_used() in order to
have !is_first_fetch() when retrying.
2018-07-23 17:50:20 +03:00
Marko Mäkelä
b660261be1 ut_print_buf_hex(): Correctly dump the hex
This should affect at least rec_printer() output.
2018-07-23 17:49:01 +03:00
Marko Mäkelä
73af075366 Reduce the number of rw_lock_own() calls
Replace pairs of rw_lock_own() calls with
calls to rw_lock_own_flagged().

These calls are only present in debug builds.
2018-07-23 17:49:01 +03:00
Marko Mäkelä
1cc1d0429d MDEV-16664: Change the default to innodb_lock_schedule_algorithm=fcfs
The parameter innodb_lock_schedule_algorithm was introduced in
MariaDB Server 10.1.19, 10.2.13, 10.3.4 as part of MDEV-11039.
In MariaDB 10.1, the default value of the parameter is 'fcfs',
that is, the existing algorithm is used by default. But in
later versions of MariaDB Server, the parameter was 'vats',
enabling the new algorithm.

Because the new algorithm is triggering a debug assertion failure
that suggests corruption of the transactional lock data structures,
we will revert to the old algorithm by default until we have
resolved the problem.
2018-07-07 11:34:26 +03:00
Thirunarayanan Balathandayuthapani
8b0d4cff07 MDEV-15855 Deadlock between purge thread and DDL statement
Problem:
========
Truncate operation holds MDL on the table (t1) and tries to
acquire InnoDB dict_operation_lock. Purge holds dict_operation_lock
and tries to acquire MDL on the table (t1) to evaluate virtual
column expressions for indexed virtual columns.
It leads to deadlock of purge and truncate table (DDL).

Solution:
=========
If purge tries to acquire MDL on the table then it should do the following:

i) Purge should release all innodb latches (including dict_operation_lock)
before acquiring metadata lock on the table.

ii) After acquiring metadata lock on the table, it should check whether the
table was dropped or renamed. If the table is dropped then purge should
ignore the undo log record. If the table is renamed then it should
release the old MDL and acquire MDL on the new name.

iii) Once purge acquires MDL, it should use the SQL table handle for all
the remaining virtual index for the purge record.

purge_node_t: Introduce new virtual column information to know whether
the MDL was acquired successfully.

This is joint work with Marko Mäkelä.
2018-07-06 17:13:53 +03:00
Thirunarayanan Balathandayuthapani
d897b4dc25 MDEV-15855: Use atomics for dict_table_t::n_ref_count
Make dict_table_t::n_ref_count private, and protect it with
a combination of dict_sys->mutex and atomics. We want to be
able to invoke dict_table_t::release() without holding
dict_sys->mutex.
2018-07-05 15:47:51 +03:00
Marko Mäkelä
e9f1d8da57 Fix warnings about possibly uninitialized variables 2018-07-05 14:14:31 +03:00
Vladislav Vaintroub
b71c9ae030 amend fix for MDEV-16596 - do not use CREATE_NEW flag when reopening redo log file.
use OPEN_ALWAYS instead, since we know file already exist.
2018-07-01 14:00:29 +01:00
Vladislav Vaintroub
c612a1e77c MDEV-16596 : Windows - redo log does not work on native 4K sector disks.
Disks with native 4K sectors need 4K alignment and size for  unbuffered IO
(i.e for files opened with FILE_FLAG_NO_BUFFERING)

Innodb opens redo log with FILE_FLAG_NO_BUFFERING, however it always does
512byte IOs. Thus, the IO on 4K native sectors will fail, rendering
Innodb non-functional.

The fix is to check whether OS_FILE_LOG_BLOCK_SIZE is multiple of logical
sector size, and if it is not, reopen the redo log without
FILE_FLAG_NO_BUFFERING flag.
2018-06-30 11:04:51 +01:00
Vladislav Vaintroub
04677f44c7 Innodb : do not use errno on Windows to print os_file_pwrite() error.
Use GetLastError() instead.
2018-06-28 17:23:05 +01:00
Marko Mäkelä
377cd52064 Pretty-print table names in some error messages 2018-06-27 23:22:08 +03:00
Marko Mäkelä
31c950cca8 Merge 10.1 into 10.2 2018-06-26 18:16:49 +03:00
Marko Mäkelä
c6392d52ee Merge 10.0 into 10.1 2018-06-26 17:34:44 +03:00
Marko Mäkelä
1b4ac075bf Merge 10.1 into 10.2 2018-06-26 15:39:23 +03:00
Marko Mäkelä
c4eb4bcef6 MDEV-16515 InnoDB: Failing assertion: ++retries < 10000 in file
dict0dict.cc

buf_LRU_drop_page_hash_for_tablespace(): Return whether any adaptive
hash index entries existed. If yes, the caller should keep retrying to
drop the adaptive hash index.

row_import_for_mysql(), row_truncate_table_for_mysql(),
row_drop_table_for_mysql(): Ensure that the adaptive hash index was
entirely dropped for the table.
2018-06-26 11:34:51 +03:00
Eugene Kosov
ff8b3c8df8 MDEV-15953 Alter InnoDB Partitioned Table Moves Files (which were originally not in the datadir) to the datadir
ha_innobase::prepare_inplace_alter_table: preserve DATA DICTIONARY for table
2018-06-26 11:11:15 +03:00
Sergei Golubchik
a40c06e03a Merge branch 'connect/10.2' into 10.2 2018-06-24 15:27:38 +02:00
Sergei Golubchik
f577311027 fix mroonga post-install script
it only worked if mroonga plugin wasn't installed before (normal case),
but then it didn't need to delete anything.

if, by some glitch, mroonga was already installed, it would delete
mroonga from mysql.plugin, but INSTALL would fail (as mroonga was running),
and the script aborted, leaving mroonga not in mysql.plugin at all.
2018-06-24 15:18:36 +02:00
Sergei Golubchik
ef64856b97 don't crash on innodb_undo_tablespaces=1 2018-06-21 23:49:37 +02:00
Sergei Golubchik
b942aa34c1 Merge branch '10.1' into 10.2 2018-06-21 23:47:39 +02:00
Sergei Golubchik
5f0510225a MDEV-16238 root/localhost authn prioritizes authentication_string over Password
Don't let SET PASSWORD to set the password, if auth_string is set.

Now SET PASSWORD always sets the plugin/auth_string fields and clears
the password field (on pre-plugin mysql.user table it works as before).
2018-06-21 10:15:27 +02:00
Sergei Golubchik
1033fa4bcc MDEV-13403 Mariadb (with TokuDB) excessive memory usage/leak
In RPM/DEB packages - always ld-preload jemalloc, instead
of linking ha_tokudb.so with it.

Keep linking in bintars, because they don't install cnf files
in the correct locations.
2018-06-21 10:15:27 +02:00
Alexander Barkov
4b821e02f6 Merge remote-tracking branch 'origin/10.0' into 10.1 2018-06-20 16:57:21 +04:00
Alexander Barkov
6c08ff3eb7 Merge remote-tracking branch 'origin/5.5' into 10.0 2018-06-20 16:55:24 +04:00
Alexander Barkov
170b43c156 MDEV-16534 PPC64: Unexpected error with a negative value into auto-increment columns in HEAP, MyISAM, ARIA 2018-06-20 16:36:46 +04:00
Vicențiu Ciorbaru
d79bf0009a MDEV-16525: MyRocks linking fails with: Undefined reference to `ZDICT_trainFromBuffer'
RocksDB will only build with libzstd support if libzstd version is
>=1.1.13. Unfortunately CMake's FindPackage claims it found version
1.1.13 when we have 1.1.12-1 installed, so a workaround with
CheckFunctionExists is used to properly check for correct libzstd
support.
2018-06-20 01:23:07 +03:00
Monty
ab19466656 MDEV-15114 ASAN heap-use-after-free in mem_heap_dup or dfield_data_is_binary_equal
The bug was that innobase_get_computed_value() trashed record[0] and data
in Field_blob::value

Fixed by using a record on the heap for innobase_get_computed_value()

Reviewer: Marko Mäkelä
2018-06-19 16:23:34 +03:00
Alexander Barkov
f5b128dfad Merge remote-tracking branch 'origin/10.0' into 10.1 2018-06-19 14:04:53 +04:00
Alexander Barkov
c450f7d8d5 Merge remote-tracking branch 'origin/5.5' into 10.0 2018-06-19 14:03:41 +04:00
Olivier Bertrand
c69efab396 - Fix MDEV-16167 Cannot insert unsigned values into a VEC table
modified:   storage/connect/filamvct.cpp
  modified:   storage/connect/tabvct.cpp

_ Typo
    modified:   storage/connect/CMakeLists.txt
2018-06-18 17:54:30 +02:00
Thirunarayanan Balathandayuthapani
eb77f8cf8d MDEV-16087 Inconsistent SELECT results when query cache is enabled
The following conditions will decide the query cache retrieval or
storing inside innodb:

    (1) There should not be any locks on the table.
    (2) Some other trx shouldn't invalidated the cache before the
    transaction started.
    (3) Read view shouldn't exist. If exists then the view
    low_limit_id should be greater than or equal to the transaction that
    invalidates the cache for the particular table.

 For read-only transaction: should satisfy  the above (1) and (3)
 For read-write transaction: should satisfy the above (1), (2), (3).

 - Changed the variable from query_cache_inv_id to query_cache_inv_trx_id.

 - Moved the function row_search_check_if_query_cache_permitted from
    row0sel.h and made it as static function in ha_innodb.cc
2018-06-18 14:26:37 +05:30
Marko Mäkelä
b8514c94f6 MDEV-16496 Mariabackup: Implement --verbose option to instrument InnoDB log apply
srv_print_verbose_log: Introduce the value 2 to refer to
mariabackup --verbose.

recv_recover_page(), recv_parse_log_recs(): Add output for
mariabackup --verbose.
2018-06-15 16:14:12 +03:00
Jan Lindström
c69357d8d4 MDEV-15611 Due to the failure of foreign key detection, Galera slave node killed himself.
Merge following change from 10.2

revision-id: d52cff9f10aeea208a1058f7b5527e602125584c (mariadb-10.2.14-25-gd52cff9)
parent(s): bc2501453c
author: Sachin Setiya
committer: Sachin Setiya
timestamp: 2018-04-04 12:26:06 +0530
message:

MDEV-15611 Due to the failure of foreign key detection, Galera...
slave node killed himself.

Problem:- If we try to delete table with foreign key and table whom it is
referring with wsrep_slave_threads>1 then galera tries to execute both
Delete_rows_log-event in parallel, which should not happen.

Solution:- This is happening because we do not have foreign key info in
write set. Upto version 10.2.7 it used to work fine. Actually it happening
because of issue in commit 2f342c4. wsrep_must_process_fk should be used
with negation.
2018-06-15 09:04:26 +03:00
Marko Mäkelä
2ca904f0ca MDEV-13103 Deal with page_compressed page corruption
fil_page_decompress(): Replaces fil_decompress_page().
Allow the caller detect errors. Remove
duplicated code. Use the "safe" instead of "fast" variants of
decompression routines.

fil_page_compress(): Replaces fil_compress_page().
The length of the input buffer always was srv_page_size (innodb_page_size).
Remove printouts, and remove the fil_space_t* parameter.

buf_tmp_buffer_t::reserved: Make private; the accessors acquire()
and release() will use atomic memory access.

buf_pool_reserve_tmp_slot(): Make static. Remove the second parameter.
Do not acquire any mutex. Remove the allocation of the buffers.

buf_tmp_reserve_crypt_buf(), buf_tmp_reserve_compression_buf():
Refactored away from buf_pool_reserve_tmp_slot().

buf_page_decrypt_after_read(): Make static, and simplify the logic.
Use the encryption buffer also for decompressing.

buf_page_io_complete(), buf_dblwr_process(): Check more failures.

fil_space_encrypt(): Simplify the debug checks.

fil_space_t::printed_compression_failure: Remove.

fil_get_compression_alg_name(): Remove.

fil_iterate(): Allocate a buffer for compression and decompression
only once, instead of allocating and freeing it for every page
that uses compression, during IMPORT TABLESPACE. Also, validate the
page checksum before decryption, and reduce the scope of some variables.

fil_page_is_index_page(), fil_page_is_lzo_compressed(): Remove (unused).

AbstractCallback::operator()(): Remove the parameter 'offset'.
The check for it in FetchIndexRootPages::operator() was basically
redundant and dead code since the previous refactoring.
2018-06-14 14:23:01 +03:00
Marko Mäkelä
f5eb37129f MDEV-13103 Deal with page_compressed page corruption
fil_page_decompress(): Replaces fil_decompress_page().
Allow the caller detect errors. Remove
duplicated code. Use the "safe" instead of "fast" variants of
decompression routines.

fil_page_compress(): Replaces fil_compress_page().
The length of the input buffer always was srv_page_size (innodb_page_size).
Remove printouts, and remove the fil_space_t* parameter.

buf_tmp_buffer_t::reserved: Make private; the accessors acquire()
and release() will use atomic memory access.

buf_pool_reserve_tmp_slot(): Make static. Remove the second parameter.
Do not acquire any mutex. Remove the allocation of the buffers.

buf_tmp_reserve_crypt_buf(), buf_tmp_reserve_compression_buf():
Refactored away from buf_pool_reserve_tmp_slot().

buf_page_decrypt_after_read(): Make static, and simplify the logic.
Use the encryption buffer also for decompressing.

buf_page_io_complete(), buf_dblwr_process(): Check more failures.

fil_space_encrypt(): Simplify the debug checks.

fil_space_t::printed_compression_failure: Remove.

fil_get_compression_alg_name(): Remove.

fil_iterate(): Allocate a buffer for compression and decompression
only once, instead of allocating and freeing it for every page
that uses compression, during IMPORT TABLESPACE.

fil_node_get_space_id(), fil_page_is_index_page(),
fil_page_is_lzo_compressed(): Remove (unused code).
2018-06-14 13:46:07 +03:00
Marko Mäkelä
72005b7a1c MDEV-13103: Improve 'cannot be decrypted' error message
buf_page_check_corrupt(): Display the file name.
2018-06-13 16:02:40 +03:00
Marko Mäkelä
3fcc11fbb4 Remove traces of the non-working MDEV-6354
MariaDB never supported the MySQL 5.7 compression format.

FIL_PAGE_TYPE_COMPRESSED: Remove. This was originally added
as FIL_PAGE_COMPRESSED.
2018-06-13 16:02:40 +03:00
Marko Mäkelä
92bd177fe9 Correct a typo in a comment 2018-06-13 16:02:40 +03:00
Sergei Petrunia
8662015c90 MDEV-15304: Server crash in print_keydup_error / key_unpack or unexpected ER_DUP_KEY
Adjust the patch to match the variant accepted into the upstream:
undo the changes in ha_rocksdb::load_hidden_pk_value().
2018-06-13 15:26:50 +03:00