Commit graph

183620 commits

Author SHA1 Message Date
Sergei Petrunia
e058a251c1 MDEV-17518: Range optimization doesn't use ON expressions from nested outer joins
Part#2: take into account that join nest that we are marking as constant
might already have constant tables in it. Don't count these tables twice.
2018-11-05 21:10:15 +03:00
Sergei Petrunia
f0cf85fd15 MDEV-17518: Range optimization doesn't use ON expressions from nested outer joins
Update .result files after the previous patch
2018-11-05 21:10:15 +03:00
Sergei Petrunia
14b62b1578 MDEV-17518: Range optimization doesn't use ON expressions from nested outer joins
Continuation of the fix: Make condition selectivity estimate use the
right estimate, too.
2018-11-05 21:10:15 +03:00
Sergei Petrunia
3b6d903852 MDEV-17493: Partition pruning doesn't work for nested outer joins
Reuse the fix for MDEV-17518 here, too.
2018-11-05 21:10:15 +03:00
Sergei Petrunia
03680a9b4f MDEV-17518: Range optimization doesn't use ON expressions from nested outer joins 2018-11-05 21:10:15 +03:00
Marko Mäkelä
a33c0e3f34 Minor clean-up for MDEV-17038
ha_innobase::check_if_supported_inplace_alter(): Remove a redundant
condition and defer some computations.

If INNOBASE_ALTER_VERSIONED_REBUILD is set, innobase_need_rebuild()
will necessarily hold. Therefore, it is not necessary to assign
need_rebuild at the start of the function.
2018-11-01 17:30:11 +02:00
Aleksey Midenkov
4acfc6ecd9 MDEV-17038 ALTER TABLE CHANGE COLUMN c1 c1 bigint NOT NULL - generates error if table uses SYSTEM VERSIONING
* Fine-grained inplace skipping by INNOBASE_ALTER_VERSIONED_REBUILD;
* Fixed column WITHOUT SYSTEM VERSIONING + ADD COLUMN;
* Fixed instant field change (MDEV-16330);
* Revisited test versioning.online;
* Merged the test versioning.trx_id_versioning_attribute_persistence
  to versioning.online;
* Renamed some versioning functions:
** change_fields_versioning_cache() -> vers_change_fields_cache()
** change_field_versioning_try() -> vers_change_field_try()
Skip condition moved out of func.

Closes 
Closes 
Related to 
2018-11-01 16:42:06 +02:00
Monty
d30124e844 MDEV-17503 CREATE SEQUENCE failed with innodb_force_primary_key =1
Fixed by adding table flag HA_WANTS_PRIMARY_KEY, which is like
HA_REQUIRE_PRIMARY_KEY but tells SQL upper layer that the storage engine
internally can handle tables without primary keys (for example for
sequences or trough user variables)
2018-10-29 16:12:52 +02:00
Monty
6a6cc8a653 Remove not used table_flag HA_NO_VARCHAR 2018-10-29 15:48:49 +02:00
Sergei Petrunia
cfd81de6d4 MDEV-17414: MyROCKS order desc limit 1 fails
- Use the correct range bounds when doing a reverse-ordered range scan
  (this was already done for HA_READ_PREFIX_LAST_OR_PREV but not for
  HA_READ_BEFORE_KEY).
2018-10-29 13:46:19 +03:00
Alexander Barkov
3e47b41a16 MDEV-17542 10.3: gcc-8.0 produces lots of -Wclass-memaccess warnings in Table_scope_and_contents_source_st 2018-10-25 14:07:31 +04:00
Vladislav Vaintroub
554ce5a0cc Attempt to fix build by a workaround for a bug in Windows8.1 SDK
Problem description and workaround are here
https://developercommunity.visualstudio.com/content/problem/185399/error-c2760-in-combaseapih-with-windows-sdk-81-and.html
2018-10-20 12:56:28 +01:00
Marko Mäkelä
5b63a660dd MDEV-11369/MDEV-12288: Reset DB_TRX_ID on the metadata record
On the hidden metadata record, if instant ALTER TABLE was executed
multiple times on the same table, purge could fail to reset the
DB_TRX_ID,DB_ROLL_PTR on the updated metadata record. This is
only a cosmetic failure that was caught (and separately fixed)
in 10.4 during the MDEV-15562 development. The problem was that
occasionally, innodb.instant_alter_crash would fail with a
result difference due to the DB_TRX_ID,DB_ROLL_PTR not having
been reset on the metadata record.

This bug should have no noticeable impact, because the metadata
record is invisible to the SQL layer, and never subjected to
MVCC or locking.

I was unable to repeat the problem on 10.3.

row_purge_parse_undo_rec(): Set node->ref for the metadata record.
2018-10-20 09:58:34 +03:00
wlad
d6889f2b58 MDEV-17504 : add diagnostics if creation of directories fail in mysql_install_db.exe
In addition,don't remove datadir if it was not created or was not empty.
2018-10-19 21:58:28 +01:00
Marko Mäkelä
1595ff8a2c Merge 10.2 into 10.3 2018-10-19 09:32:52 +03:00
Marko Mäkelä
ab1ce2204e MDEV-17466: Remove the debug assertion
This reverts commit 2d4075e1d9
where the debug assertion was added. There seems to be a potential
problem in the purge of indexes that depend on virtual columns.

Ultimately, we should change the InnoDB undo log format so that
all actual secondary index keys are stored there, also for
virtual or spatial indexes. In that way, purge and rollback would
be more straightforward.
2018-10-19 09:31:33 +03:00
Marko Mäkelä
abbf169f52 Remove unused TIMETPF 2018-10-19 09:28:21 +03:00
halfspawn
67f06cadc3 MDEV-17359 Concatenation operator || in like expression 2018-10-18 14:13:00 +04:00
Marko Mäkelä
981a81090f Merge 10.2 into 10.3 2018-10-18 12:15:31 +03:00
Marko Mäkelä
64d26ec52a Fix the Windows build
On Windows, a mismatch between TIMETPF ("%ld") and time_t would be reported.
Use "%ld" and long, like the code used to be.
2018-10-18 12:15:07 +03:00
Marko Mäkelä
22eb146454 Merge 10.2 into 10.3 2018-10-18 11:27:28 +03:00
Marko Mäkelä
3eaae09669 Extend the use of innodb_default_row_format.combinations 2018-10-18 11:27:19 +03:00
Marko Mäkelä
56de291978 Fix a merge error in commit 28f08d3753 2018-10-18 10:10:11 +03:00
Marko Mäkelä
f454189c60 Merge 10.2 into 10.3 2018-10-17 19:37:05 +03:00
Marko Mäkelä
2fa4ed031c MDEV-17483 Insert on delete-marked record can wrongly inherit old values for instantly added column
row_ins_clust_index_entry_low(): Do not call dtuple_t::trim()
before row_ins_clust_index_entry_by_modify(), so that the values
of all columns will be available in row_upd_build_difference_binary().
If applicable, the tuple can be trimmed in btr_cur_optimistic_update()
or btr_cur_pessimistic_update(), which will be called by
row_ins_clust_index_entry_by_modify().
2018-10-17 18:55:46 +03:00
Marko Mäkelä
853a0a4368 MDEV-13564: Set innodb_safe_truncate=ON by default
The setting innodb_safe_truncate=ON reduces compatibility with older
versions of MariaDB and backup tools in two ways.

First, we will be writing TRX_UNDO_RENAME_TABLE records, which older
versions do not know about. These records could be misinterpreted if
a DDL transaction was recovered and would be rolled back.
Such rollback is only possible if the server was killed while
an incomplete DDL transaction was persisted. On transaction completion,
the insert_undo log pages would only be repurposed for new undo log
allocations, and their contents would not matter. So, older versions
will not have a problem with innodb_safe_truncate=ON if the server was
shut down cleanly.

Second, to prevent such recovery failure, innodb_safe_truncate=ON will
cause a modification of the redo log format identifier, which will
prevent older versions from starting up after a crash. MariaDB Server
versions older than 10.2.13 will refuse to start up altogether, even
after clean shutdown.

A server restart with innodb_safe_truncate=OFF will restore compatibility
with older server and backup versions.
2018-10-17 17:35:38 +03:00
Igor Babaev
c2c1550f57 MDEV-17419 Subquery with group by returns wrong results
Added only test case because the bug was fixed by the patch for mdev-17382.
2018-10-17 04:37:25 -07:00
Igor Babaev
dc3f55947e MDEV-17107 Assertion `table_list->table' failed in find_field_in_table_ref
Added only test case because the bug was fixed by the patch for mdev-16992.
2018-10-17 03:26:30 -07:00
Varun Gupta
97a37edc97 MDEV-17137: Syntax errors with VIEW using MEDIAN
The syntax error happened because we had not implemented a different print for
percentile functions. The syntax is a bit different when we use percentile functions
as window functions in comparision to normal window functions.
Implemented a seperate print function for percentile functions
2018-10-16 08:11:26 -07:00
Andrei Elkin
2308b9afec MDEV-17098 DATE -> DATETIME replication conversion not working, even in ALL_NON_LOSSY mode
Opened up MYSQL_TYPE _DATETIME{,2} <-> _NEWDATE conversions for replication.
2018-10-16 12:56:33 +03:00
Marko Mäkelä
2d4075e1d9 MDEV-17466 Virtual column value not available during purge
row_build_index_entry_low(): Assert that when the value of a
virtual column is not available, this can only happen when
the index creation was completed but not committed yet.

This change is not fixing any bug, making a debug assertion
stricter, so that bugs can be caught in the future.

Ultimately, we should change the InnoDB undo log format so that
all actual secondary index keys are stored there, also for
virtual or spatial indexes. In that way, purge and rollback would
be more straightforward.
2018-10-16 10:42:30 +03:00
Igor Babaev
103b1df510 MDEV-17222 Reproducible server crash in String_list::append_str or
in Field_iterator_table::create_item

When IN predicate is converted to IN subquery we have to ensure that
any item from the select list of the subquery has some name and this name
is unique across the select list.
This was not guaranteed by the code before the patch for MDEV-17222.
If the name of an item of the select list was not set, and this happened
for binary constants, then the server crashed. If the first row in the IN
list contained the same constant in two different positions then the server
returned an error message.
This was fixed by providing all constants in the first row of the IN list
with generated names.
2018-10-14 15:29:08 -07:00
Varun Gupta
af6077b535 MDEV-16990:server crashes in base_list_iterator::next
When we have a query which has implicit_grouping then we are sure that we would end up with only one
row so there is no point to do DISTINCT computation
2018-10-14 10:44:00 -07:00
Kristian Nielsen
74387028a0 Merge branch 'gtid_table_garbage_rows' into 10.3 2018-10-13 23:48:43 +02:00
Kristian Nielsen
61bba2a540 Fix build of embedded server 2018-10-13 22:53:12 +02:00
Kristian Nielsen
00164ea4b1 Merge branch 'gtid_table_garbage_rows_10.3' into 10.3 2018-10-13 19:50:53 +02:00
Elena Stepanova
b715a0fe45 Disabled storage_engine test for RocksDB due to unstable execution plan
The failure caused by MDEV-16387
2018-10-13 19:14:31 +03:00
Igor Babaev
6d29c8527b MDEV-17354 Server crashes in add_key_field / .. / Item_func_null_predicate::add_key_fields
upon INSERT .. SELECT

The function Item *Item_direct_view_ref::derived_field_transformer_for_where()
erroneously did not strip off ref wrappers from references to materialized
derived tables / views. As a result the expressions that contained some
references of the type Item_direct_view_ref to columns of a materialized
derived table / view V were pushed into V incorrectly. This could cause
crashes for some INSERT ... SELECT statements.
2018-10-12 11:45:04 -07:00
Vladislav Vaintroub
2fd770641e Revert the last change to replication test
disable rpl_mdev382 on Windows, due to unix shell escaping used there.
2018-10-12 20:07:08 +02:00
Vladislav Vaintroub
00c40efcd6 Fix portability issues with rpl test suite. 2018-10-12 16:43:45 +01:00
Alexander Barkov
6120ae4aca Adjusting old tests and adding new tests for "MDEV-8765: mysqldump -use utf8mb4 by default" 2018-10-12 06:34:13 +04:00
Daniel Black
ce643ddac7 MDEV-8765: mysqldump -use utf8mb4 by default
Bump mysqldump to a version greater than that on the 10.3 branch.
2018-10-12 05:16:01 +04:00
Marko Mäkelä
7e869a2767 Merge 10.2 into 10.3 2018-10-11 23:09:10 +03:00
Marko Mäkelä
81a5b6ccd5 MDEV-17433 Allow InnoDB start up with empty ib_logfile0 from mariabackup --prepare
A prepared backup from Mariabackup does not really need to contain any
redo log file, because all log will have been applied to the data files.

When the user copies a prepared backup to a data directory (overwriting
existing files), it could happen that the data directory already contained
redo log files from the past. mariabackup --copy-back) would delete the
old redo log files, but a user’s own copying script might not do that.
To prevent corruption caused by mixing an old redo log file with data
files from a backup, starting with MDEV-13311, Mariabackup would create
a zero-length ib_logfile0 that would prevent startup.

Actually, there is no need to prevent InnoDB from starting up when a
single zero-length file ib_logfile0 is present. Only if there exist
multiple data files of different lengths, then we should refuse to
start up due to inconsistency. A single zero-length ib_logfile0 should
be treated as if the log files were missing: create new log files
according to the configuration.

open_log_file(): Remove. There is no need to open the log files
at this point, because os_file_get_status() already determined
the size of the file.

innobase_start_or_create_for_mysql(): Move the creation of new
log files a little later, not when finding out that the first log
file does not exist, but after finding out that it does not exist
or it exists as a zero-length file.
2018-10-11 23:00:48 +03:00
Marko Mäkelä
b8944e8972 Fix a sign mismatch 2018-10-11 22:47:42 +03:00
Marko Mäkelä
6319c0b541 MDEV-13564: Replace innodb_unsafe_truncate with innodb_safe_truncate
Rename the 10.2-specific configuration option innodb_unsafe_truncate
to innodb_safe_truncate, and invert its value.

The default (for now) is innodb_safe_truncate=OFF, to avoid
disrupting users with an undo and redo log format change within
a Generally Available (GA) release series.
2018-10-11 15:10:13 +03:00
Alexander Barkov
4de0d920be MDEV-17411 Wrong WHERE optimization with simple CASE and searched CASE 2018-10-11 13:39:53 +04:00
Marko Mäkelä
30629e196d MDEV-13564: Null-merge 10.2 into 10.3
We keep the MySQL 5.7 backup-incompatible TRUNCATE TABLE
only in MariaDB Server 10.2. In 10.3 and later releases,
only the backup-friendly TRUNCATE will be available.
2018-10-11 08:33:59 +03:00
Marko Mäkelä
ae9d82c9f8 Merge 10.2 into 10.3 2018-10-11 08:22:08 +03:00
Marko Mäkelä
3448ceb02a MDEV-13564: Implement innodb_unsafe_truncate=ON for compatibility
While MariaDB Server 10.2 is not really guaranteed to be compatible
with Percona XtraBackup 2.4 (for example, the MySQL 5.7 undo log format
change that could be present in XtraBackup, but was reverted from
MariaDB in MDEV-12289), we do not want to disrupt users who have
deployed xtrabackup and MariaDB Server 10.2 in their environments.

With this change, MariaDB 10.2 will continue to use the backup-unsafe
TRUNCATE TABLE code, so that neither the undo log nor the redo log
formats will change in an incompatible way.

Undo tablespace truncation will keep using the redo log only. Recovery
or backup with old code will fail to shrink the undo tablespace files,
but the contents will be recovered just fine.

In the MariaDB Server 10.2 series only, we introduce the configuration
parameter innodb_unsafe_truncate and make it ON by default. To allow
MariaDB Backup (mariabackup) to work properly with TRUNCATE TABLE
operations, use loose_innodb_unsafe_truncate=OFF.

MariaDB Server 10.3.10 and later releases will always use the
backup-safe TRUNCATE TABLE, and this parameter will not be
added there.

recv_recovery_rollback_active(): Skip row_mysql_drop_garbage_tables()
unless innodb_unsafe_truncate=OFF. It is too unsafe to drop orphan
tables if RENAME operations are not transactional within InnoDB.

LOG_HEADER_FORMAT_10_3: Replaces LOG_HEADER_FORMAT_CURRENT.

log_init(), log_group_file_header_flush(),
srv_prepare_to_delete_redo_log_files(),
innobase_start_or_create_for_mysql(): Choose the redo log format
and subformat based on the value of innodb_unsafe_truncate.
2018-10-11 08:17:04 +03:00