Commit graph

69353 commits

Author SHA1 Message Date
Marko Mäkelä
46facaedbf Fix GCC 9 -Wmaybe-uninitialized
Always initialize ScopedStatementReplication::saved_binlog_format,
so that GCC cannot emit a bogus warning about
ScopedStatementReplication::~ScopedStatementReplication() using the
variable.

The code was originally introduced in
commit d998da0306.
2019-09-26 15:43:51 +03:00
Alexander Barkov
b6bb64e54a Merge remote-tracking branch 'origin/10.2' into 10.3 2019-09-24 23:05:09 +04:00
Alexander Barkov
0a359d7627 Merge remote-tracking branch 'origin/10.1' into 10.2 2019-09-24 19:19:25 +04:00
Marko Mäkelä
7ae290c757 Merge 10.2 into 10.3 2019-09-24 13:39:31 +03:00
Alexander Barkov
5f118b26c8 Merge remote-tracking branch 'origin/5.5' into 10.1 2019-09-24 11:18:39 +04:00
Alexander Barkov
67b0faa29e MDEV-20495 Assertion `precision > 0' failed in decimal_bin_size upon CREATE .. SELECT with zerofilled decimal
Also fixes:

MDEV-20560 Assertion `precision > 0' failed in decimal_bin_size upon SELECT with MOD short unsigned decimal

Changing the way how Item_func_mod calculates its max_length.
It now uses decimal_precision(), decimal_scale() and unsigned_flag
of its arguments, like all other Item_num_op descendants do.
2019-09-24 10:46:18 +04:00
Varun Gupta
ab9f378b0b Backporting 273d8eb12c Proper fix for disabling warnings in read_statistics_for_table() 2019-09-23 14:11:48 +05:30
Michael Widenius
e157f81771 Remove Field::is_stat_field completely 2019-09-23 10:42:20 +03:00
Marko Mäkelä
c016ea660e Merge 10.2 into 10.3 2019-09-23 10:25:34 +03:00
Marko Mäkelä
44c5144943 Merge 10.1 into 10.2 2019-09-23 08:26:08 +03:00
Michael Widenius
1bbe8c5e0f Proper fix for disabling warnings in read_statistics_for_table().
MDEV-20589: Server still crashes in Field::set_warning_truncated_wrong_value

- Use dbug_tmp_use_all_columns() to mark that all fields can be used
- Remove field->is_stat_field (not needed)
- Remove extra arguments to Field::clone() that should not be there
- Safety fix for Field::set_warning_truncated_wrong_value() to not crash
  if table is zero in production builds (We have got crashes several times
  here so better to be safe than sorry).
- Threat wrong character string warnings identical to other field
  conversion warnings. This removes some warnings we before got from
  internal conversion errors.  There is no good reason why a user would
  get an error in case of 'key_field='wrong-utf8-string' but not for
  'field=wrong-utf8-string'.  The old code could also easily give
  thousands of no-sence warnings for one single statement.
2019-09-22 04:08:48 +03:00
Varun Gupta
896974fc3d MDEV-18094: Query with order by limit picking index scan over filesort
In the function test_if_cheaper_ordering we make a decision if using an index is better than
using filesort for ordering. If we chose to do range access then in test_quick_select we
should make sure that cost for table scan is set to DBL_MAX so that it is not picked.
2019-09-21 12:14:05 +05:30
Igor Babaev
ba7725dace MDEV-20229 CTE defined with table value constructor cannot be used in views
A CTE can be defined as a table values constructor. In this case the CTE is
always materialized in a temporary table.
If the definition of the CTE contains a list of the names of the CTE
columns then the query expression that uses this CTE can refer to the CTE
columns by these names. Otherwise the names of the columns are taken from
the names of the columns in the result set of the query that specifies the
CTE.
Thus if the column names of a CTE are provided in the definition the
columns of result set should be renamed. In a general case renaming of
the columns is done in the select lists of the query specifying the CTE.
If a CTE is specified by a table value constructor then there are no such
select lists and renaming is actually done for the columns of the result
of materialization.
Now if a view is specified by a query expression that uses a CTE specified
by a table value constructor saving the column names of the CTE in the
stored view definition becomes critical: without these names the query
expression is not able to refer to the columns of the CTE.

This patch saves the given column names of CTEs in stored view definitions
that use them.
2019-09-20 15:59:54 -07:00
Anel Husakovic
1ad79c8187 MDEV-19679 - CREATE SERVER needs tweaks for compatibility with CONNECT engine 2019-09-20 01:54:38 -07:00
Marko Mäkelä
bb4214272a Merge 10.1 into 10.2 2019-09-18 16:24:48 +03:00
Varun Gupta
273d8eb12c MDEV-20589: Server still crashes in Field::set_warning_truncated_wrong_value
The flag is_stat_field is not set for the min_value and max_value of field items
inside table share. This is a must requirement as we don't want to throw
warnings of truncation when we read values from the statistics table to the column
statistics of table share fields.
2019-09-18 15:06:02 +05:30
Sujatha
90a9c4cae7 MDEV-20217: Semi_sync: Last_IO_Error: Fatal error: Failed to run 'after_queue_event' hook
Fix:
===
Implemented upstream fix.

commit 7d3d0fc303
Author: He Zhenxing <zhenxing.he@sun.com>

Backport Bug#45852 Semisynch: Last_IO_Error: Fatal error: Failed
to run 'after_queue_event' hook

Errors when send reply to master should never cause the IO thread
to stop, because master can fall back to async replication if it
does not get reply from slave.

The problem is fixed by deliberately ignoring the return value of
slave_reply.
2019-09-16 15:45:24 +05:30
Teemu Ollakka
40beeb1402 MDEV-20561 Galera node shutdown fails in non-Primary (#1386)
Command COM_SHUTDOWN was rejected in non-Primary because
server_command_flags[COM_SHUTDOWN] had value CF_NO_COM_MULTI
instead of CF_SKIP_WSREP_CHECK.

As a fix removed assignment
server_command_flags[CF_NO_COM_MULTI]= CF_NO_COM_MULTI
which overwrote server_command_flags[COM_SHUTDOWN].
2019-09-13 09:18:11 +03:00
Igor Babaev
deb9121fdf MDEV-20576 A new assertion added to check validity of calculated
selectivity values fails

After having set the assertion that checks validity of selectivity values
returned by the function table_cond_selectivity() a test case from
order_by.tesst failed. The failure occurred because range optimizer could
return as an estimate of the cardinality of the ranges built for an index
a number exceeding the total number of records in the table.

The second bug is more subtle. It may happen when there are several
indexes with same prefix defined on the first joined table t accessed by
a constant ref access. In this case the range optimizer estimates the
number of accessed records of t for each usable index and these
estimates can be different. Only the first of these estimates is taken
into account when the selectivity of the ref access is calculated.
However the optimizer later can choose a different index that provides
a different estimate. The function table_condition_selectivity() could use
this estimate to discount the selectivity of the ref access. This could
lead to an selectivity value returned by this function that was greater
that 1.
2019-09-12 23:01:11 -07:00
Sergei Petrunia
c8dc866fde MDEV-20371: Invalid reads at plan refinement stage: join->positions...
best_access_path() is called from two optimization phases:

1. Plan choice phase, in choose_plan(). Here, the join prefix being
   considered is in join->positions[]

2. Plan refinement stage, in fix_semijoin_strategies_for_picked_join_order
   Here, the join prefix is in join->best_positions[]

It used to access join->positions[] from stage #2. This didnt cause any
valgrind or asan failures (as join->positions[] has been written-to before)
but the effect was similar to that of reading the random data:
The join prefix we've picked (in join->best_positions) could have
nothing in common with the join prefix that was last to be considered
(in join->positions).
2019-09-11 17:06:50 +03:00
Marko Mäkelä
c6a6830916 Merge 10.2 into 10.3 2019-09-11 10:32:24 +03:00
Alexander Barkov
f1616bacb7 Adding missing semicolons to sql_yacc_ora.yy (10.3), indentation cleanups. 2019-09-11 10:13:19 +04:00
Marko Mäkelä
f7fe51f126 Merge 10.1 into 10.2 2019-09-11 08:48:00 +03:00
Marko Mäkelä
d6f0e60a67 Merge 5.5 into 10.1
Null merge the MDEV-14383 fix; it affects 5.5 only.
2019-09-11 08:11:46 +03:00
Alexander Barkov
f1309fac33 Adding missing semicolons to sql_yacc.yy (10.3), indentation cleanups. 2019-09-11 05:12:37 +04:00
Alexander Barkov
48f8e3f3f4 Merge remote-tracking branch 'origin/10.2' into 10.3 2019-09-11 04:47:01 +04:00
Alexander Barkov
5ec4efb7b1 Adding missing semicolons to sql_yacc.yy (10.2) 2019-09-11 04:44:24 +04:00
Marko Mäkelä
ff5ecfd335 Correct the merge 0f83c8878d
Re-enable some Galera tests that should have been enabled.

Add client_ed25519.so to debian/libmariadb3.install;
merge e47a143fc0 correctly.

Remove a duplicated #include from wsrep_mysqld.cc.
2019-09-10 10:04:04 +03:00
Marko Mäkelä
da9201dd5b Merge 10.2 into 10.3 2019-09-10 09:25:20 +03:00
Nikita Malyavin
aabd1c8fcb MDEV-16490 fix versioning.partition failure 2019-09-09 22:30:53 +03:00
Nikita Malyavin
f6a7730c45 MDEV-16490: It's possible to make a system versioned table without any versioning field
* do not allow versioned table to be without versioned (non-system) fields
* prohibit changing field versioning, when removing table versioning
* handle CREATE...SELECT as well
2019-09-09 20:14:47 +03:00
Vladislav Vaintroub
2336e0b394 MDEV-20206 : Crash inside timer_callback()[threadpool_win.cc:419]
The most likely cause of the crash is that a timer fired, after it was closed.

MSDN documents such a possibility, in the documentation for
CloseThreadpoolTimer() function, and recommends disabling the timer before
calling WaitForThreadpoolTimerCallbacks()/CloseThreadpoolTimer().

The fix follows this recommendation.

Note, that 5.5-10.1 disabled the timer before close, but this code
was lost in threadpool refactoring in 10.2
2019-09-09 13:52:30 +02:00
Vladislav Vaintroub
803bb5cdf8 Windows, cmake : Fix occasional link error when switching between debug to optimized compilation
For Visual Studio generator, use a per-config .def/.lib files with symbols
exported from mysqld.exe

Functions exported from mysqld.exe may differ between debug/optimized
compilation, e.g dbug functions are missing in release config.
2019-09-09 13:52:30 +02:00
Alexey Botchkov
031c695b8c MDEV-16594 ALTER DATA DIRECTORY in PARTITIONS of InnoDB storage does nothing silently
InnoDB intentionally (it's a documented behavior) ignores changing of
DATA DIRECTORY and INDEX DIRECTORY for partitions. Though we should
issue warning when this happens.
2019-09-09 15:39:12 +04:00
Jan Lindström
0fd5b11eb0 MDEV-20511: Galera replication of events is not consistent
After SST from master node (the one where event is ENABLED) - you will end up with the event enabled on two nodes, hence it's now being executed twice. It can be solved by comparing event's originator with server_id. if not equal, then change its status to 'SLAVESIDE_DISABLED'

Changes to be committed:
	new file:   mysql-test/suite/galera/r/galera_events2.result
	new file:   mysql-test/suite/galera/t/galera_events2.test
	modified:   sql/events.cc
2019-09-09 14:23:59 +03:00
Alexander Barkov
8750df43ab MDEV-20517 Assertion `!is_expensive()' failed in Item::value_depends_on_sql_mode_const_item 2019-09-07 07:44:54 +04:00
Sergei Golubchik
f80e02e043 Merge branch '10.2' into 10.3 2019-09-06 16:58:39 +02:00
Sergei Golubchik
5a9e2b77d4 Part3: MDEV-18156 Assertion 0' failed or btr_validate_index(index, 0, false)' in row_upd_sec_index_entry or error code 126: Index is corrupted upon DELETE with PAD_CHAR_TO_FULL_LENGTH
Don't break compatibility in GA releases.
Warn the user, but don't refuse to create a table until 10.5
2019-09-06 16:35:56 +02:00
Marko Mäkelä
537f8594a6 Merge 10.2 into 10.3 2019-09-04 17:52:04 +03:00
Sergei Golubchik
8dca4cf53f MDEV-20403 Assertion 0' or Assertion btr_validate_index(index, 0)' failed in row_upd_sec_index_entry or error code 126: Index is corrupted upon UPDATE with TIMESTAMP..ON UPDATE
remove a special treatment of a bare DEFAULT keyword that made it
behave inconsistently and differently from DEFAULT(column).
Now all forms of the explicit assignment of a default column value
behave identically, and all count as an explicitly assigned value
(for the purpose of ON UPDATE NOW).

followup for c7c481f4d9
2019-09-04 15:37:23 +02:00
Sergei Golubchik
647d5b2430 MDEV-20079 When setting back the system time while mysqld is running, NOW() and UNIX_TIMESTAMP() results get stuck
typo. system_time.start wasn't updated when system_time.sec
and system_time.sec_part were.
2019-09-04 09:30:43 +02:00
Sergei Golubchik
08b01ace8a MDEV-16871 in_predicate_conversion_threshold cannot be set in my.cnf 2019-09-04 09:30:43 +02:00
Marko Mäkelä
b07beff894 MDEV-15326: InnoDB: Failing assertion: !other_lock
MySQL 5.7.9 (and MariaDB 10.2.2) introduced a race condition
between InnoDB transaction commit and the conversion of implicit
locks into explicit ones.

The assertion failure can be triggered with a test that runs
3 concurrent single-statement transactions in a loop on a simple
table:

CREATE TABLE t (a INT PRIMARY KEY) ENGINE=InnoDB;
thread1: INSERT INTO t SET a=1;
thread2: DELETE FROM t;
thread3: SELECT * FROM t FOR UPDATE; -- or DELETE FROM t;

The failure scenarios are like the following:
(1) The INSERT statement is being committed, waiting for lock_sys->mutex.
(2) At the time of the failure, both the DELETE and SELECT transactions
are active but have not logged any changes yet.
(3) The transaction where the !other_lock assertion fails started
lock_rec_convert_impl_to_expl().
(4) After this point, the commit of the INSERT removed the transaction from
trx_sys->rw_trx_set, in trx_erase_lists().
(5) The other transaction consulted trx_sys->rw_trx_set and determined
that there is no implicit lock. Hence, it grabbed the lock.
(6) The !other_lock assertion fails in lock_rec_add_to_queue()
for the lock_rec_convert_impl_to_expl(), because the lock was 'stolen'.
This assertion failure looks genuine, because the INSERT transaction
is still active (trx->state=TRX_STATE_ACTIVE).

The problematic step (4) was introduced in
mysql/mysql-server@e27e0e0bb7
which fixed something related to MVCC (covered by the test
innodb.innodb-read-view). Basically, it reintroduced an error
that had been mentioned in an earlier commit
mysql/mysql-server@a17be6963f:
"The active transaction was removed from trx_sys->rw_trx_set prematurely."

Our fix goes along the following lines:

(a) Implicit locks will released by assigning
trx->state=TRX_STATE_COMMITTED_IN_MEMORY as the first step.
This transition will no longer be protected by lock_sys_t::mutex,
only by trx->mutex. This idea is by Sergey Vojtovich.
(b) We detach the transaction from trx_sys before starting to release
explicit locks.
(c) All callers of trx_rw_is_active() and trx_rw_is_active_low() must
recheck trx->state after acquiring trx->mutex.
(d) Before releasing any explicit locks, we will ensure that any activity
by other threads to convert implicit locks into explicit will have ceased,
by checking !trx_is_referenced(trx). There was a glitch
in this check when it was part of lock_trx_release_locks(); at the end
we would release trx->mutex and acquire lock_sys->mutex and trx->mutex,
and fail to recheck (trx_is_referenced() is protected by trx_t::mutex).
(e) Explicit locks can be released in batches (LOCK_RELEASE_INTERVAL=1000)
just like we did before.

trx_t::state: Document that the transition to COMMITTED is only
protected by trx_t::mutex, no longer by lock_sys_t::mutex.

trx_rw_is_active_low(), trx_rw_is_active(): Document that the transaction
state should be rechecked after acquiring trx_t::mutex.

trx_t::commit_state(): New function to change a transaction to committed
state, to release implicit locks.

trx_t::release_locks(): New function to release the explicit locks
after commit_state().

lock_trx_release_locks(): Move much of the logic to the caller
(which must invoke trx_t::commit_state() and trx_t::release_locks()
as needed), and assert that the transaction will have locks.

trx_get_trx_by_xid(): Make the parameter a pointer to const.

lock_rec_other_trx_holds_expl(): Recheck trx->state after acquiring
trx->mutex, and avoid a redundant lookup of the transaction.

lock_rec_queue_validate(): Recheck impl_trx->state while holding
impl_trx->mutex.

row_vers_impl_x_locked(), row_vers_impl_x_locked_low():
Document that the transaction state must be rechecked after
trx_mutex_enter().

trx_free_prepared(): Adjust for the changes to lock_trx_release_locks().
2019-09-04 09:42:38 +03:00
Alexander Barkov
7e08ac0b41 Merge 10.2 (up to commit ef00ac4c86) into 10.3 2019-09-04 10:19:58 +04:00
Sergei Golubchik
c7c481f4d9 MDEV-20403 Assertion 0' or Assertion btr_validate_index(index, 0)' failed in row_upd_sec_index_entry or error code 126: Index is corrupted upon UPDATE with TIMESTAMP..ON UPDATE
Three issues here:
* ON UPDATE DEFAULT NOW columns were updated after generated columns
  were computed - this broke indexed virtual columns
* ON UPDATE DEFAULT NOW columns were updated after BEFORE triggers,
  so triggers didn't see the correct NEW value
* in case of a multi-update generated columns were also updated
  after BEFORE triggers
2019-09-03 20:34:30 +02:00
Sergei Golubchik
3789692d17 don't compare unassigned columns
on UPDATE, compare_record() was comparing all columns that are marked
for writing. But generated columns that are written to the table are
always deterministic and cannot change unless normal non-generated
columns were changed. So it's enough to compare only non-generated
columns that were explicitly assigned values in the SET clause.
2019-09-03 20:34:30 +02:00
Sergei Golubchik
17ab02f4b0 cleanup: on update default now
* remove one level of virtual functions
* remove redundant checks
* remove an if() as the value is always known at compilation time

don't pretend that "DEFAULT expr" and "ON UPDATE DEFAULT NOW"
are "basically the same thing"
2019-09-03 20:34:30 +02:00
Monty
a071e0e029 Merge branch '10.2' into 10.3 2019-09-03 13:17:32 +03:00
Alexander Barkov
ef00ac4c86 Part2: MDEV-18156 Assertion 0' failed or btr_validate_index(index, 0, false)' in row_upd_sec_index_entry or error code 126: Index is corrupted upon DELETE with PAD_CHAR_TO_FULL_LENGTH
This patch allows the server to open old tables that have
"bad" generated columns (i.e. indexed virtual generated columns,
persistent generated columns) that depend on sql_mode,
for general things like SELECT, INSERT, DROP, etc.
Warning are issued in such cases.

Only these commands are now disallowed and return an error:
- CREATE TABLE introducing a "bad" generated column
- ALTER TABLE introducing a "bad" generated column
- CREATE INDEX introdicing a "bad" generated column
  (i.e. adding an index on a virtual generated column
   that depends on sql_mode).

Note, these commands are allowed:
- ALTER TABLE removing a "bad" generate column
- ALTER TABLE removing an index from a "bad" virtual generated column
- DROP INDEX removing an index from a "bad" virtual generated column
but only if the table does not have any "bad" columns as a result.
2019-09-03 09:51:35 +04:00
Alexander Barkov
dc719597ee MDEV-18156 Assertion 0' failed or btr_validate_index(index, 0, false)' in row_upd_sec_index_entry or error code 126: Index is corrupted upon DELETE with PAD_CHAR_TO_FULL_LENGTH
This change takes into account a column's GENERATED ALWAYS AS
expression dependcy on sql_mode's PAD_CHAR_TO_FULL_LENGTH and
NO_UNSIGNED_SUBTRACTION flags.

Indexed virtual columns as well as persistent generated columns are
now not allowed to have such dependencies to avoid inconsistent data
or index files on sql_mode changes.
So an error is now returned in cases like this:

  CREATE OR REPLACE TABLE t1
  (
    a CHAR(5),
    v VARCHAR(5) AS (a) PERSISTENT -- CHAR->VARCHAR or CHAR->TEXT = ERROR
  );

Functions RPAD() and RTRIM() can now remove dependency on
PAD_CHAR_TO_FULL_LENGTH. So this can be used instead:

  CREATE OR REPLACE TABLE t1
  (
    a CHAR(5),
    v VARCHAR(5) AS (RTRIM(a)) PERSISTENT
  );

Note, unlike CHAR->VARCHAR and CHAR->TEXT this still works,
not RPAD(a) is needed:

  CREATE OR REPLACE TABLE t1
  (
    a CHAR(5),
    v CHAR(5) AS (a) PERSISTENT -- CHAR->CHAR is OK
  );

More sql_mode flags may affect values of generated columns.
They will be addressed separately.

See comments in sql_mode.h for implementation details.
2019-09-03 05:34:53 +04:00