Commit graph

21631 commits

Author SHA1 Message Date
Kentoku
6d80d35d91 MDEV-16787 optimistic parallel replication fails on spider
Add a system variable spider_slave_trx_isolation.
- spider_slave_trx_isolation
  The transaction isolation level when Spider table is used by slave SQL thread.
  -1 : OFF
   0 : READ UNCOMMITTED
   1 : READ COMMITTED
   2 : REPEATABLE READ
   3 : SERIALIZABLE
  The default value is -1

Miscellaneous Spider typos
2019-01-31 08:51:34 +01:00
Kentoku
bef6b197fc MDEV-16520 Out-Of-Memory running big aggregate query on Spider Engine
Change default value of the followings
quick_mode 0 -> 3
quick_page_size 100 -> 1024

Add the following parameter for limiting result page size by byte
- quick_page_byte(qpb)
  Number of bytes in a page when acquisition one by one.
  When quick_mode is 1 or 2, Spider stores at least 1 record even if
  quick_page_byte is smaller than 1 record. When quick_mode is 3,
  quick_page_byte is used for judging using temporary table.
  That is given to priority when server parameter spider_quick_page_byte
  is set.
  The default value is 10485760

Fix "out of sync" issue at using quick_mode = 1 or 2
2019-01-31 08:51:34 +01:00
Kentoku
3cb7c5f27b MDEV-16279 Spider crashes on CHECKSUM TABLE with spider_quick_mode=3
The fields of the temporary table were not created in create_tmp_table function. Because item->const_item() was true. But the temporary tables that is created by Spider are always used all columns. So Spider should call create_tmp_table function with TMP_TABLE_ALL_COLUMNS flag.
2019-01-31 08:51:34 +01:00
Kentoku
6caf9ec425 Update Spider to version 3.3.14. Add direct left outer join/right outer join/inner join feature 2019-01-31 08:51:34 +01:00
Thirunarayanan Balathandayuthapani
b8aef87221 MDEV-16849 Extending indexed VARCHAR column should be instantaneous
Analysis:
========
Increasing the length of the indexed varchar column is not an instant operation for
innodb.

Fix:
===
- Introduce the new handler flag 'Alter_inplace_info::ALTER_COLUMN_INDEX_LENGTH' to
indicate the index length differs due to change of column length changes.

- InnoDB makes the ALTER_COLUMN_INDEX_LENGTH flag as instant operation.

This is a port of Mysql fix.

    commit 913071c0b16cc03e703308250d795bc381627e37
    Author: Nisha Gopalakrishnan <nisha.gopalakrishnan@oracle.com>
    Date:   Wed May 30 14:54:46 2018 +0530

        BUG#26848813: INDEXED COLUMN CAN'T BE CHANGED FROM VARCHAR(15)
                      TO VARCHAR(40) INSTANTANEOUSLY
2019-01-30 15:33:32 +05:30
Marko Mäkelä
1522ee2949 MDEV-18016: Assertion failure on ALTER TABLE after foreign_key_checks=0
ha_innobase::commit_inplace_alter_table(): Do not crash if
innobase_update_foreign_cache() returns an error. It can return
an error on ALTER TABLE if an inconsistent FOREIGN KEY constraint
was created earlier when SET foreign_key_checks=0 was in effect.
Instead, report a warning to the client that constraints cannot
be loaded.
2019-01-29 15:20:26 +02:00
Marko Mäkelä
6699cac0bf MDEV-18256 Duplicated call to dict_foreign_remove_from_cache()
ha_innobase::prepare_inplace_alter_table(): Filter out duplicates
from ha_alter_info->alter_info->drop_list.elements.
2019-01-29 15:20:26 +02:00
Marko Mäkelä
5e06ee41a4 MDEV-18222: Duplicated call to dict_foreign_remove_from_cache()
innobase_rename_column_try(): Declare fk_evict as std::set
instead of std::list, in order to filter out duplicates.
2019-01-29 15:20:26 +02:00
Oleksandr Byelkin
f877f6b49d Fix xtradb version after merge 2019-01-29 11:50:07 +01:00
Oleksandr Byelkin
c991939bab MariaDB detect incorrect table name 2019-01-29 09:34:08 +01:00
Oleksandr Byelkin
8c2f3e0c16 Fix detection of version in tokudb 2019-01-28 20:17:54 +01:00
Oleksandr Byelkin
5cdb3fb25e Merge branch 'merge-tokudb-5.6' into 10.0 2019-01-28 20:15:57 +01:00
Marko Mäkelä
36be0a5aef MDEV-18399 Recognize the deprecated parameters innodb_file_format, innodb_large_prefix
The parameters innodb_file_format and innodb_large_prefix were overridden
in the Debian-distributed configuration files, because the default values
of these parameters between MariaDB 5.5 and MariaDB 10.2
did not make any sense.

To allow a more seamless upgrade from MariaDB 10.1 to later versions,
let InnoDB recognize the parameters innodb_file_format and
innodb_large_prefix and issue deprecation warnings for them if they
are specified. A deprecation period of only one major release
(one year between the MariaDB 10.2 and 10.3 releases) is insufficient
for these widely used parameters.
2019-01-28 17:58:14 +02:00
Oleksandr Byelkin
724b09d5e7 Version fix after merge 2019-01-28 15:42:16 +01:00
Oleksandr Byelkin
94b68b35f4 Reverting part of da34c7de5d that was already fixed by MDEV-17531 by Marko 2019-01-28 15:39:27 +01:00
Oleksandr Byelkin
959f7415bd Merge branch 'merge-xtradb-5.6' into 10.0 2019-01-28 15:37:01 +01:00
Jan Lindström
97930df13c
Merge pull request from codership/10.2-MDEV-15740
MDEV-15740 Fixes to Galera transaction recovery
2019-01-28 12:01:35 +02:00
Olivier Bertrand
bece44be4f - Enable CONNECT tables to have triggers
Update version number
  modified:   storage/connect/ha_connect.cc
2019-01-27 14:21:39 +01:00
Honza Horak
7334f9717d Do not import commands library as it is not used
Closes 
2019-01-25 20:00:00 +01:00
Eugene Kosov
0e1f7f5c4a MDEV-18057 Assertion `(node->state == 5) || (node->state == 6)' failed in row_upd_sec_step upon DELETE after UPDATE failed due to FK violation
The idea of the fix: reset state from previous query.

row_upd_clust_step(): reset cached index before updating a clustered index

Closes 
2019-01-25 16:35:50 +02:00
Marko Mäkelä
9bd80ada6f Merge 10.2 into 10.3 2019-01-25 16:35:13 +02:00
Teemu Ollakka
040b840de7 MDEV-15740 Backport wsrep recovery fixes from 10.4.
Clear wsrep XID in innobase_rollback_by_xid() for recovered wsrep
transaction in order to avoid resetting XID storage when rolling back
wsrep transaction during recovery.

Sort wsrep XIDs read from storage engine in ascending order and
erify that the range is continuous during crash recovery. If binlog is off,
commit all recovered transactions for continuous seqno range. This is safe
because all transactions with wsrep XID have been certified and must be
committed in the cluster. On the other hand if binlog is on, respect binlog
as a transaction coordinator in order to avoid missing transactions in binlog
that have been committed into storage engine .
2019-01-25 16:19:20 +02:00
Eugene Kosov
31d0727a10 MDEV-18235: Changes related to fsync()
Remove fil_node_t::sync_event.

I had a discussion with kernel fellows and they said it's safe to call
fsync() simultaneously at least on VFS and ext4. So initially I wanted
to disable check for recent Linux but than I realized code is buggy.

Consider a case when one thread is inside fsync() and two others are
waiting inside os_event. First thread after fsync() calls os_event_set()
which is a broadcast! So two waiting threads will awake and may call
fsync() at the same time.

One fix is to add a notify_one() functionality to os_event but I decided
to remove incorrect check completely. Note, it works for one waiting
thread but not for more than one.

IMO it's ok to avoid existing bugs but there is not too much sense in
avoiding possible(!) bugs as this code does.

fil_space_t::is_in_rotation_list(), fil_space_t::is_in_unflushed_spaces():
Replace redundant bool fields with member functions.

fil_node_t::needs_flush: Replaces fil_node_t::modification_counter and
fil_node_t::flush_counter. We need to know whether there _are_ some
unflushed writes and we do not need to know _how many_ writes.

fil_system_t::modification_counter: Remove as not needed.
Even if we needed fil_node_t::modification_counter, every file
could have its own counter that would be incremented on each write.

fil_system_t::modification_counter is a global modification counter
for all files. It was incremented on every write. But whether some
file was flushed or not is an internal fil_node_t deal/state and
this makes fil_system_t::modification_counter useless.

Closes 
2019-01-25 15:40:04 +02:00
Olivier Bertrand
bcb8a52295 - Fix MDEV-18192: CONNECT Engine JDBC not able to issue
simple UPDATE statement from trigger or stored procedure
  modified:   storage/connect/tabext.cpp
  modified:   storage/connect/tabext.h
  modified:   storage/connect/tabjdbc.cpp

- Make user and password defined in CREATE TABLE have precedence on
  the ones specified in a Federated Server.
  modified:   storage/connect/tabjdbc.cpp

- JSONColumns: Copy locally constant strings to fix error in OEM modules
  modified:   storage/connect/tabjson.cpp
2019-01-25 13:02:40 +01:00
Olivier Bertrand
e6fcd72309 Squashed commit of connect/10.0:
commit 6a6a1f37798
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Fri Jan 4 12:31:52 2019 +0100

    - Fix a few bug mainly concerning discovery and call from OEM
      (and prepare new table types)

      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/tabjson.h
      modified:   storage/connect/tabxml.cpp
      modified:   storage/connect/tabxml.h

    - Fix wrong line estimate
      modified:   storage/connect/mysql-test/connect/r/part_table.result
      modified:   storage/connect/mysql-test/connect/t/part_table.test

commit bd7d2e912d9
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Tue Dec 4 23:35:09 2018 +0100

    Fix wrong version number

commit 4933680e7ab
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Sun Dec 2 00:25:05 2018 +0100

    - Make PlugSubAlloc to be exportable
      Suppress unused parameter from PlugSubSet
      modified:   storage/connect/global.h
      modified:   storage/connect/plugutil.cpp
      modified:   storage/connect/jsonudf.cpp
      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/user_connect.cc

    - Fix a bug making column catalog XML tables fail
      modified:   storage/connect/tabxml.cpp

    - Comment out wrong message
      modified:   storage/connect/ha_connect.cc

    - Update error message when sorting an ODBC table fails
      modified:   storage/connect/tabodbc.cpp

    - Add error message when gettting an address
      from an OEM fails.
      modified:   storage/connect/reldef.cpp

    - Make some modifications useful for OEM module writting
      Export discovery functions for CSV, JDBC and XML
      Remove unuseful include from tabjson.h
      Move TDBXML::data_charset function from header file to source
      modified:   storage/connect/tabfmt.h
      modified:   storage/connect/tabjson.h
      modified:   storage/connect/tabxml.cpp
      modified:   storage/connect/tabxml.h

    - Update test result
      modified:   storage/connect/mysql-test/connect/r/jdbc_oracle.result
2019-01-25 10:06:16 +01:00
Oleksandr Byelkin
13802fef83 5.6.42-84.2 2019-01-24 17:31:13 +01:00
Oleksandr Byelkin
3262afc6c5 5.6.42-84.2 2019-01-24 16:48:39 +01:00
Marko Mäkelä
947b6b849d Merge 10.2 into 10.3 2019-01-24 16:14:12 +02:00
Marko Mäkelä
46f712c73c MDEV-15114: Fix memory leaks
When innobase_allocate_row_for_vcol() returns true (for failure),
it may already have invoked mem_heap_create(). However, some callers
would fail to invoke mem_heap_free().
2019-01-24 12:32:27 +02:00
Marko Mäkelä
d283f80eae Update the InnoDB version number 2019-01-23 19:46:35 +02:00
Aditya A
aa8a31dadd Bug GCOLS: INCORRECT BEHAVIOR AFTER DATA INSERTED WITH IGNORE KEYWORD
PROBLEM
-------

1. We are inserting a base column entry which causes an invalid value
   by the function provided to generate virtual column,but we go ahead
   and insert this due to ignore keyword.
2. We then delete this record, making this record delete marked in innodb.
   If we try to insert another record with the same pk as the deleted
   record and if the rec is not purged ,then we try to undelete mark this
   record and try to build a update vector with previous and updated value
   and while calculating the value of virtual column we get error from
   server that we cannot calculate this from base column.
   Innodb assumes that innobase_get_computed_value() Should always return
   a valid value for the base column present in the row. The failure of
   this call was not handled ,so we were crashing.

FIX
2019-01-23 19:46:35 +02:00
Marko Mäkelä
9a7281a703 Merge 10.1 into 10.2 2019-01-23 15:09:06 +02:00
Marko Mäkelä
d06ebd932d Remove references to removed dict_sys->size 2019-01-23 14:42:21 +02:00
Marko Mäkelä
2565c02ca5 Remove unnecessary type casts 2019-01-23 14:42:21 +02:00
Marko Mäkelä
3b6d2efcb1 Merge 10.0 into 10.1 2019-01-23 14:34:23 +02:00
Marko Mäkelä
52d13036d8 MDEV-17933 slow server status - dict_sys_get_size()
dict_sys_get_size(): Replace the time-consuming loop with
a crude estimate that can be computed without holding any mutex.

Even before dict_sys->size was removed in MDEV-13325,
not all memory allocations by the InnoDB data dictionary cache
were being accounted for. One example is foreign key constraints.
Another example is virtual column metadata, starting with 10.2.
2019-01-23 13:44:20 +02:00
Sergei Golubchik
2a0f1d6132 Bug#28867993: POSSIBLE ISSUE WITH MYSQL SERVER RESTART
on startup innodb is checking whether files "ib_logfileN"
(for N from 1 to 100) exist, and whether they're readable.
A non-existent file aborted the scan.
A directory instead of a file made InnoDB to fail.

Now it treats "directory exists" as "file doesn't exist".
2019-01-23 12:39:03 +01:00
Marko Mäkelä
31d592ba7d MDEV-18349 InnoDB file size changes are not safe when file system crashes
When InnoDB is invoking posix_fallocate() to extend data files, it
was missing a call to fsync() to update the file system metadata.
If file system recovery is needed, the file size could be incorrect.

When the setting innodb_flush_method=O_DIRECT_NO_FSYNC
that was introduced in MariaDB 10.0.11 (and MySQL 5.6) is enabled,
InnoDB would wrongly skip fsync() after extending files.

Furthermore, the merge commit d8b45b0c00
inadvertently removed XtraDB error checking for posix_fallocate()
which this fix is restoring.

fil_flush(): Add the parameter bool metadata=false to request that
fil_buffering_disabled() be ignored.

fil_extend_space_to_desired_size(): Invoke fil_flush() with the
extra parameter. After successful posix_fallocate(), invoke
os_file_flush(). Note: The bookkeeping for fil_flush() would not be
updated the posix_fallocate() code path, so the "redundant"
fil_flush() should be a no-op.
2019-01-23 12:18:00 +02:00
Jan Lindström
55be043c13
Merge pull request from tempesta-tech/sysprg/MDEV-17835v2
MDEV-17835: Remove wsrep-sst-method=xtrabackup
2019-01-23 12:07:36 +02:00
Marko Mäkelä
942a6bd009 MDEV-18349 InnoDB file size changes are not safe when file system crashes
fil_extend_space_to_desired_size(): Invoke fsync() after posix_fallocate()
in order to durably extend the file in a crash-safe file system.
2019-01-23 09:51:06 +02:00
Sergey Vojtovich
9f4d4f404f MDEV-12747 - main.mysqld_option_err fails in buildbot with timeout
thd_destructor_proxy() may miss abort signal if innobase_end() is running
concurrently, which causes server hang in pthread_join() on shutdown.

The problem was that aborting wasn't protected by mutex:
proxy thr: while (!myvar->abort)
end thr: running->abort = 1;
end thr: mysql_cond_broadcast(...);
proxy thr: mysql_cond_wait(...); // nobody to awake it
end thr: pthread_join(...); // waits for proxy thr

Also made main.mysqld_option_err reentrant.
2019-01-22 18:26:37 +04:00
Julius Goryavsky
0e89e90f42 MDEV-17835: Remove wsrep-sst-method=xtrabackup
The second line of changes related to replacing xtrabackup with
mariabackup:

1) All unnecessary references to xtrabackup are removed from
the documentation, from some comments, from the control files
that are used to prepare the packages.

2) Made corrections of the tests from the galera_3nodes suite
that mentioned xtrabackup or the old (associated with xtrabackup)
version of innobackupex.

3) Fixed flaws in the galera_3nodes mtr suite control scripts,
because of which they could not work with mariabackup.

4) Fixed numerous bugs in the SST scripts and in the mtr test
files (galera_3nodes mtr suite) that prevented the use of Galera
with IPv6 addresses.

5) Fixed flaws in tests for rsync and mysqldump (for galera_3nodes
mtr tests suite). These tests were not performed successfully without
these fixes.

https://jira.mariadb.org/browse/MDEV-17835
2019-01-22 13:28:03 +01:00
Marko Mäkelä
d4144c8e01 MDEV-17821 Assertion !page_rec_is_supremum(rec) failed in btr_pcur_store_position
MDEV-11369 (instant ADD COLUMN) introduced a regression in the case the
leftmost leaf page of the clustered index is empty except for the
hidden metadata record.

btr_pcur_store_position(): If the only record in the leftmost leaf page
is the metadata record, store the position before the first user record
in the tree.
2019-01-21 09:50:52 +02:00
Marko Mäkelä
778192454e MDEV-17823: Fix the non-debug build 2019-01-18 14:23:04 +02:00
Marko Mäkelä
4e75bfcb21 MDEV-18152 Assertion 'num_fts_index <= 1' failed
InnoDB does not allow creating multiple FULLTEXT INDEX
in ALGORITHM=INPLACE. This constraint was not being properly
enforced after MariaDB started to support ALGORITHM=INSTANT
and instant ADD COLUMN.

As a side effect of this bug, we again allow ALGORITHM=INPLACE
to rebuild a table when one FULLTEXT INDEX survives.
Also, we are returning a more accurate reason for refusing LOCK=NONE.

innobase_fulltext_exist(): Return the number of fulltext indexes.

ha_innobase::check_if_supported_inplace_alter(): If the table
needs to be rebuilt, refuse the operation if multiple fulltext
indexes would remain.
2019-01-18 12:39:19 +02:00
Marko Mäkelä
a0d3ead83a Merge 10.2 into 10.3 2019-01-18 10:43:31 +02:00
Marko Mäkelä
5f60c7c3c2 MDEV-17823 Assertion failed when accessing indexed instantly added column
This assertion would fail when a secondary index record for an
instantly added column was accessed.

It is unclear to me why this code path is executed so rarely.
I was unable to cover it even when using FORCE INDEX.

row_sel_sec_rec_is_for_clust_rec(): Remove the assertion, and use
the proper function rec_get_nth_cfield().

row_sel_store_mysql_field_func(): Simply use rec_get_nth_cfield()
instead of duplicating its logic.
2019-01-18 10:39:52 +02:00
Marko Mäkelä
37ffdb44ef Merge 10.1 into 10.2 2019-01-18 06:51:52 +02:00
Marko Mäkelä
1d72db45a8 MDEV-18237 InnoDB: Unable to drop FTS index aux table and further errors (possibly bogus)
row_drop_table_for_mysql(): Fix a regression introduced in MDEV-16515.
Similar to the follow-up fixes MDEV-16647 and MDEV-17470, we must make
the internal tables of FULLTEXT INDEX immune to kills, to avoid noise
and resource leakage on DROP TABLE or ALTER TABLE. (Orphan internal tables
would be dropped at the next InnoDB startup only.)
2019-01-18 06:46:39 +02:00
Vladislav Vaintroub
69221746db MDEV-18289 - Fix a race between thd_destructor_proxy() startup and
server shutdown.

Prior to adding current thd to the global thd list, initialize
thd->mysys_var->current_cond / thd->mysys_var->current_mutex.

Otherwise thd_destructor_proxy can miss the abort condition, which is set
by the shutdown thread.
2019-01-17 23:18:50 +01:00