Commit graph

182367 commits

Author SHA1 Message Date
Sergei Golubchik
dfd6702a29 MDEV-16157 federated corrupts timestamps
do "set time_zone='+00:00'" on remote server
after connecting.

temporarily reset local time zone to my_tz_OFFSET0
whenever Field::val_str() or Field::store()
methods are used.
2018-05-17 15:33:39 +02:00
Sergei Golubchik
28dbdf3d79 MDEV-14551 Can't find record in table on multi-table update with ORDER BY
preserve positions if the multi-update join is using tmp table:

* store positions in the tmp table if needed
  JOIN::add_fields_for_current_rowid()

* take positions from the tmp table, not from file->position():
  multi_update::prepare2()
2018-05-17 15:13:47 +02:00
Sergei Golubchik
e17e798599 cleanup: simplify multi-update's juggling with positions
introduce Item_temptable_rowid() that is used to store
table->file->position() in the temporary table record
2018-05-17 15:13:47 +02:00
Sergei Golubchik
c368878fac cleanup: comments and whitespaces 2018-05-17 15:13:47 +02:00
Sergei Golubchik
4b061ec4ea bugfix: correct list assignment operator 2018-05-17 15:13:47 +02:00
Monty
c2352c45fb MDEV-14669 Assertion `file->trn == trn' failed in ha_maria::start_stmt
Can't repeat issue so I just added the test case
2018-05-16 21:32:14 +03:00
Alexander Barkov
48d7038861 Token precedence cleanup in *.yy
We'll be fixing soon shift-reduce conflicts introduced in the new
10.3 syntax (see MDEV-15818 for details) by defining precedence for
a number of tokens (e.g. TIMESTAMP, TRANSACTION_SYM, TEXT_STRING)
and adding "%prec" directives.

Before doing this, it's better to have the existing precedences set properly,
for easier readability and maintainability.

Details:
- Changing precedence of NOT to its proper position (between AND and IS).
  It was wrong. It worked fine only because the relevant grammar reside
  in different separate rules (expr and predicate).

- Moving NOT2_SYM and BINARY to the same line with NEG and ~
  It worked fine because operators !, BINARY, ~ do not conflict
  to each other.

- Fixing associativity of NOT_SYM, NOT2_SYM, BINARY, COLLATE_SYM
  from "right" to "left". They are not dyadic (operate on a single expression
  only). So "%left" or "%right" is needed only to set precedence,
  while associativity does not matter.
  Note, it would be better to use "%precedence" for these tokens
  instead of "%left" though, but we use an old version of Bison on windows,
  which may not support %precedence yet.

This patch does not change behavior. The generated sql_yacc.cc and
sql_yacc_ora.cc are exactly the same before and after this change.
2018-05-16 21:58:28 +04:00
Alexander Barkov
c2df4e9d62 MDEV-16186 Concatenation operator || returns wrong results in sql_mode=ORACLE 2018-05-16 17:21:31 +04:00
Marko Mäkelä
66360506f0 MDEV-16168: Resurrect the record MLOG_UNDO_INIT
While the record type MLOG_UNDO_INIT feels redundant, it does save
redo log volume, and subtle changes to redo log volume appear to
make a big difference for the log_sys.mutex contention.

trx_undo_parse_page_init(): Allow type==0, which is what we write
since MDEV-12288. Parse type in a simpler way; it always was written
as a single byte.

trx_undo_page_init(): Write a MLOG_UNDO_INIT record.
2018-05-16 10:18:18 +03:00
Alexey Botchkov
a639eff594 MDEV-15813 ASAN use-after-poison in hp_hashnr upon HANDLER READ on a versioned HEAP table.
Check index capabilities before executing HANDLER READ command.
2018-05-16 09:44:22 +04:00
Marko Mäkelä
cd15e764a8 MDEV-16159 Use atomic memory access for purge_sys
Thanks to Sergey Vojtovich for feedback and many ideas.

purge_state_t: Remove. The states are replaced with
purge_sys_t::enabled() and purge_sys_t::paused() as follows:
PURGE_STATE_INIT, PURGE_STATE_EXIT, PURGE_STATE_DISABLED: !enabled().
PURGE_STATE_RUN, PURGE_STATE_STOP: paused() distinguishes these.

purge_sys_t::m_paused: Renamed from purge_sys_t::n_stop.
Protected by atomic memory access only, not purge_sys_t::latch.

purge_sys_t::m_enabled: An atomically updated Boolean that
replaces purge_sys_t::state.

purge_sys_t:🏃 Remove, because it duplicates
srv_sys.n_threads_active[SRV_PURGE].

purge_sys_t::running(): Accessor for srv_sys.n_threads_active[SRV_PURGE].

purge_sys_t::stop(): Renamed from trx_purge_stop().

purge_sys_t::resume(): Renamed from trx_purge_run().
Do not acquire latch; solely rely on atomics.

purge_sys_t::is_initialised(), purge_sys_t::m_initialised: Remove.

purge_sys_t::create(), purge_sys_t::close(): Instead of invoking
is_initialised(), check whether event is NULL.

purge_sys_t::event: Move before latch, so that fields that are
protected by latch can reside on the same cache line with latch.

srv_start_wait_for_purge_to_start(): Merge to the only caller srv_start().
2018-05-15 23:01:18 +03:00
Marko Mäkelä
442a6e6b25 MDEV-16172 Remove InnoDB 5.7 version number from MariaDB 10.3 onwards
Because the InnoDB implementation in MariaDB has diverged from MySQL,
it is not meaningful to report a MySQL version number for InnoDB
any more. Some examples include:

MariaDB 10.1 (which is based on MySQL 5.6) included encryption and
variable-size page compression before MySQL 5.7 introduced them.
MariaDB 10.2 (based on MySQL 5.7) introduced persistent AUTO_INCREMENT
(MDEV-6076) in a GA release before MySQL 8.0.
MariaDB 10.3 (based on MySQL 5.7) introduced instant ADD COLUMN
(MDEV-11369) before MySQL.

All of these features use a different implementation and file format.
Also, some features were never merged from MySQL 5.7, and thus MariaDB
is not affected by related bugs. Examples include CREATE TABLESPACE
and the reimplementation of the partitioning engine.
2018-05-15 23:01:18 +03:00
Marko Mäkelä
27f12c5d5b MDEV-16143 Assertion failure in rec_offs_get_n_alloc() due to row_trx_id_offset()
row_trx_id_offset(): Add the missing rec_offs_init() call.
This was broken by commit 97e51d24cb.
2018-05-15 23:01:18 +03:00
Marko Mäkelä
93b69825ad MDEV-16169 InnoDB: Failing assertion: !space->referenced()
Before invoking fil_space_t::acquire(), check space->is_stopping()
to ensure that the tablespace actually is accessible. This fixes
a regression introduced by MDEV-15983.

fil_space_next(): Remove some duplicated code for prev_space==NULL,
and check is_stopping() also on the first tablespace.

i_s_tablespaces_encryption_fill_table(),
i_s_tablespaces_scrubbing_fill_table(): Check is_stopping().
2018-05-15 23:01:18 +03:00
Marko Mäkelä
2b812abd1a Correct a comment 2018-05-15 23:01:18 +03:00
Oleksandr Byelkin
dd0e960f4d Fix of the test after null processing fix. 2018-05-15 19:52:28 +02:00
Alexander Barkov
2e0c23d82b Removing redundant "%left INTERVAL_SYM" and "%prec INTERVAL_SYM"
These directives did not have any affect.

All these affected lines:

1.  | INTERVAL_SYM expr interval '+' expr

2.  | INTERVAL_SYM '(' expr ',' expr ')'

3.  | INTERVAL_SYM '(' expr ',' expr ',' expr_list ')'

4.  | DATE_ADD_INTERVAL '(' expr ',' INTERVAL_SYM expr interval ')'

5.  | DATE_SUB_INTERVAL '(' expr ',' INTERVAL_SYM expr interval ')'

define the grammar unambiguously.

"bison -v sql_yacc.yy" produces exactly the same sql_yacc.output
before and after the change.
2018-05-15 18:24:27 +04:00
Sergei Petrunia
06f539dab5 Revert the following patch brought in by "Merge pull request #753 from shinnok/10.3-macfixes"
It caused compile failures:

commit 6620fbd62a
Author: Teodor Mircea Ionita <teodor@mariadb.org>
Date:   Thu May 10 12:23:35 2018 +0300

    MDEV-15778: On macOS pthread_t is opaque, requires explicit cast

    On macOS pthread id is a pointer to struct _opaque_pthread_t type,
    requires explicit cast to ulint which in turn is size_t;
    Was failing with Clang 9.1.0 Debug build on macOS 10.13.4:

    sync0policy.h:53:4: error: cannot initialize a member subobject of type 'ulint'
    (aka 'unsigned long') with an rvalue of type 'os_thread_id_t' (aka '_opaque_pthread_t *')
    m_thread_id(os_thread_id_t(ULINT_UNDEFINED))
                            ^           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    sync0policy.h:79:4: error: cannot initialize a parameter of type 'int64' (aka 'long long') with
    an rvalue of type 'os_thread_id_t' (aka '_opaque_pthread_t *')
    my_atomic_storelint(&m_thread_id, os_thread_get_curr_id());
2018-05-15 17:17:09 +03:00
Oleksandr Byelkin
0dd1ebcb27 MDEV-15576: Server crashed in Cached_item_str::cmp / sortcmp or Assertion `item->null_value' failed in Type_handler_temporal_result::make_sort_key upon SELECT with NULLIF and ROLLUP
Fixed null_value processing and is_null() usage.
2018-05-15 14:20:54 +02:00
Monty
8a9048bcf3 MDEV-16170 Server crashes in Item_null_result::type_handler on SELECT with ROLLUP
Problem was that I in a previous patch enabled ifdef:ed code intended for
10.3 but that never worked.

Reverted to original code
2018-05-15 13:51:03 +03:00
Monty
e7fc8cd683 MDEV-654 Assertion `share->now_transactional' failed in flush_log_for_bitmap on concurrent workload with Aria tables
I was able to repeat the problem with old version of randgen

Reason for crash:

- It's not safe to change share->now_transactional if there are changed
bitmaps in the pagecache as flushing these can cause redo-entries and
the bitmap flush code checks that share->now_transactional is set.

Fixed by flushing bitmaps in _ma_tmp_disable_logging_for_table() before
we set share->now_transactional to 0
2018-05-15 13:51:03 +03:00
Alexey Botchkov
e06c029849 MDEV-15465 Server crash or ASAN heap-use-after-free in Item_func_match::cleanup upon using FT search with partitioning.
Partition engine FT keys are implemented in such a way that
the FT function's cleanup() methods use table's internals.
So calling them after close_thread_tables is unsafe.
2018-05-15 14:10:19 +04:00
Sergei Petrunia
a0048378f9
Merge pull request #753 from shinnok/10.3-macfixes
MDEV-15778: 10.3 macOS build fixes
2018-05-15 12:20:16 +03:00
Sachin Setiya
9ee5406e2f MDEV-15965 Invisible columns and LOAD DATA don't work well together:... ER_WARN_TOO_FEW_RECORDS
Fix mysql_load iterator to skip invisible fields.
2018-05-15 12:38:44 +05:30
Alexander Barkov
46be31982a MDEV-16094 Crash when using AS OF with a stored function
MDEV-16100 FOR SYSTEM_TIME erroneously resolves string user variables as transaction IDs

Problem:

Vers_history_point::resolve_unit() tested item->result_type() before
item->fix_fields() was called.

- Item_func_get_user_var::result_type() returned REAL_RESULT by default.
  This caused MDEV-16100.
- Item_func_sp::result_type() crashed on assert.
  This caused MDEV-16094

Changes:
1. Adding item->fix_fields() into Vers_history_point::resolve_unit()
   before using data type specific properties of the history point
   expression.

2. Adding a new virtual method Type_handler::Vers_history_point_resolve_unit()

3. Implementing type-specific
   Type_handler_xxx::Type_handler::Vers_history_point_resolve_unit()
    in the way to:
    a. resolve temporal and general purpose string types to TIMESTAMP
    b. resolve BIT and general purpose INT types to TRANSACTION
    c. disallow use of non-relevant data type expressions in FOR SYSTEM_TIME

    Note, DOUBLE and DECIMAL data types are disallowed intentionally.
    - DOUBLE does not have enough precision to hold huge BIGINT UNSIGNED values
    - DECIMAL rounds on conversion to INT
    Both lack of precision and rounding might potentionally lead to
    very unpredictable results when a wrong transaction ID would be chosen.
    If one really wants dangerous use of DOUBLE and DECIMAL, explicit CAST
    can be used:

      FOR SYSTEM_TIME AS OF CAST(double_or_decimal AS UNSIGNED)

    QQ: perhaps DECIMAL(N,0) could still be allowed.

4. Adding a new virtual method Item::type_handler_for_system_time(),
   to make HEX hybrids and bit literals work as TRANSACTION rather
   than TIMESTAMP.

5. sql_yacc.yy: replacing the rule temporal_literal to "TIMESTAMP TEXT_STRING".
   Other temporal literals now resolve to TIMESTAMP through the new
   Type_handler methods. No special grammar needed. This removed
   a few shift/resolve conflicts.
   (TIMESTAMP related conflicts in "history_point:" will be removed separately)

6. Removing the "timestamp_only" parameter from
   vers_select_conds_t::resolve_units() and Vers_history_point::resolve_unit().
   It was a hint telling that a table did not have any TRANSACTION-aware
   system time columns, so it's OK to resolve to TIMESTAMP in case of uncertainty.
   In the new reduction it works as follows:
   - the decision between TIMESTAMP and TRANSACTION is first made
     based only on the expression data type only
   - then, in case if the expression resolved to TRANSACTION, the table
     is checked if TRANSACTION-aware columns really exist.
   This way is safer against possible ALTER TABLE statements changing
   ROW START and ROW END columns from "BIGINT UNSIGNED" to "TIMESTAMP(x)"
   or the other way around.
2018-05-15 09:33:29 +04:00
Igor Babaev
1b45ede6ab Adjusted test results after mdev-15159. 2018-05-14 19:02:06 -07:00
Jacob Mathew
86f9932e80 MDEV-16101: ADD PARTITION on table partitioned by list does not work with more than 32 list values.
This problem occured because the reorganization of the list of values when the
number of elements exceeds 32 was not handled correctly.  I have fixed the
problem by fixing the way that the list values are reorganized when the number
of list values exceeds 32.

Author:
  Jacob Mathew.

Reviewer:
  Alexey Botchkov.

Merged From:
  Branch bb-10.3-MDEV-16101
2018-05-14 15:12:40 -07:00
Igor Babaev
e74181e3c2 MDEV-15159 NULL is treated as 0 in CTE
Forced columns of recursive CTEs to be nullable. SQL standard
requires this only from recursive columns, but in our code
so far we do not differentiate between recursive and non-recursive
columns when aggregating types of the union that specifies a
recursive CTE.
2018-05-14 14:38:17 -07:00
Jacob Mathew
8e01598620 MDEV-16101: ADD PARTITION on table partitioned by list does not work with more than 32 list values.
This problem occured because the reorganization of the list of values when the
number of elements exceeds 32 was not handled correctly.  I have fixed the
problem by fixing the way that the list values are reorganized when the number
of list values exceeds 32.

Author:
  Jacob Mathew.

Reviewer:
  Alexey Botchkov.
2018-05-14 13:25:41 -07:00
Alexander Barkov
4a5e23e257 MDEV-16152 Expressions with INTERVAL return bad results in some cases 2018-05-14 11:36:22 +04:00
Michael Widenius
1cb4caa66d MDEV-15970 Crash when doing truncate on locked sequence
Problem was that we used table->s->db_type() for accessing
handlerton of opened file instead of table->file->ht

Other bug fixed:
- Ensure that we set error if reopen_tables() fails
  (This was the cause of assert)
2018-05-14 10:08:05 +03:00
Michael Widenius
be6ae0bb6b Don't report errors from open_table() twice 2018-05-14 10:08:05 +03:00
Michael Widenius
d853042b5f Added missing write_unlock() in case of errors 2018-05-14 10:08:04 +03:00
Marko Mäkelä
15419a5583 Merge 10.2 into 10.3 2018-05-12 22:14:59 +03:00
Marko Mäkelä
5e84ea9634 MDEV-12266: Remove dict_table_is_discarded()
The predicate dict_table_is_discarded() checks whether
ALTER TABLE…DISCARD TABLESPACE has been executed.

Replace most occurrences of dict_table_is_discarded() with
checks of dict_table_t::space. A few checks for the flag
DICT_TF2_DISCARDED are necessary; write them inline.

Because !is_readable() implies !space, some checks for
dict_table_is_discarded() were redundant.
2018-05-12 22:12:12 +03:00
Marko Mäkelä
c57e9835ff Replace dict_col_is_virtual(col) with col->is_virtual() 2018-05-12 22:12:12 +03:00
Marko Mäkelä
ba43914ec4 Replace dict_table_is_temporary(table) with table->is_temporary() 2018-05-12 22:12:12 +03:00
Sergei Golubchik
edaf57e966 disable galera.pxc-421 test 2018-05-12 17:36:28 +02:00
Eugene Kosov
f08b8fd58d MDEV-15980 FOR SYSTEM_TIME BETWEEN and FROM .. TO work with negative intervals
Fixed by using exactly the same filtering conditions as specified by standard
in 7.6 <table reference> General Rules
2018-05-12 10:16:46 +02:00
Aleksey Midenkov
b1e75d290e MDEV-14823, MDEV-15956 Versioning error messages fixes
MDEV-14823 Wrong error message upon selecting from a system_time partition
MDEV-15956 Strange ER_UNSUPPORTED_ACTION_ON_GENERATED_COLUMN upon ALTER on versioning column
2018-05-12 10:16:46 +02:00
Aleksey Midenkov
ce2cf855bf MDEV-16043 Assertion thd->Item_change_list::is_empty() failed in mysql_parse upon SELECT from a view reading from a versioned table
Lost restore_active_arena().
Using of Query_arena_stmt is suggested instead.
2018-05-12 10:16:46 +02:00
Sergei Golubchik
8b2fa0ab25 remove obsolete rdiff 2018-05-12 10:16:46 +02:00
Aleksey Midenkov
68cad6aa24 MDEV-16024 transaction_registry.begin_timestamp is wrong for explicit transactions
Store transaction start time in thd->transaction.start_time.
THD::transaction_time() wraps over transaction.start_time taking into
account current status of BEGIN.
2018-05-12 10:16:46 +02:00
Aleksey Midenkov
fd4153f083 MDEV-16010 Too many rows with AS OF point_in_the_past_or_NULL 2018-05-12 10:16:46 +02:00
Aleksey Midenkov
fddaaf7295 MDEV-15978 Add Feature_system_versioning status variable 2018-05-12 10:16:46 +02:00
Eugene Kosov
da25860d4a MDEV-15995 Drop extra semicolon in VARIABLE_COMMENT for SYSTEM_VERSIONING_ALTER_HISTORY 2018-05-12 10:16:46 +02:00
Eugene Kosov
fe10bf870c MDEV-15893 Rename VTQ to TRT 2018-05-12 10:16:46 +02:00
Sergei Golubchik
a4272bf154 versioning: use @@timestamp
Don't use hidden system time in versioning,
but keep the system time logic in THD
to workaround low-res system clock and
replication not versioned to versioned.

This reverts MDEV-14788 (System versioning cannot
be based on local timestamps, as it is now).
Versioning is based on local timestamps again,
but timestamps are protected by MDEV-15923
(option to control who can set session @@timestamp).
2018-05-12 10:16:46 +02:00
Sergei Golubchik
4203f572ae MDEV-15923 option to control who can set session @@timestamp
--secure-timestamp=NO|SUPER|REPLICATION|YES
2018-05-12 10:16:45 +02:00
Sergei Golubchik
bbf5cf4ddf cleanup: sys_vars.cc
remove the redundant declaration tail
2018-05-12 10:16:45 +02:00