Commit graph

9802 commits

Author SHA1 Message Date
Sergei Golubchik
44ca4993b8 bugfix: partitioning and keyread on an indexed vcol 2016-12-12 20:35:50 +01:00
Sergei Golubchik
2a0f7a34d6 bugfix: non-deterministic vcols in partitioning 2016-12-12 20:35:50 +01:00
Sergei Golubchik
d1f3763323 bugfix: non-deterministic vcols in indexes 2016-12-12 20:35:49 +01:00
Sergei Golubchik
a411d7f4f6 store/show vcols as item->print()
otherwise we'd need to store sql_mode *per vcol*
(consider CREATE INDEX...) and how SHOW CREATE TABLE would
support that?

Additionally, get rid of vcol::expr_str, just to make sure
the string is always generated and never leaked in the
original form.
2016-12-12 20:35:41 +01:00
Sergei Golubchik
8b3b6dc377 test how MDL blocks InnoDB purge
it must, because purge opens a table that might be being altered right now
2016-12-12 20:27:43 +01:00
Sergei Golubchik
1cae1af6f9 MDEV-5800 InnoDB support for indexed vcols
* remove old 5.2+ InnoDB support for virtual columns
  * enable corresponding parts of the innodb-5.7 sources
  * copy corresponding test cases from 5.7
  * copy detailed Alter_inplace_info::HA_ALTER_FLAGS flags from 5.7
     - and more detailed detection of changes in fill_alter_inplace_info()
  * more "innodb compatibility hooks" in sql_class.cc to
     - create/destroy/reset a THD (used by background purge threads)
     - find a prelocked table by name
     - open a table (from a background purge thread)

  * different from 5.7:
    - new service thread "thd_destructor_proxy" to make sure all THDs are
      destroyed at the correct point in time during the server shutdown
    - proper opening/closing of tables for vcol evaluations in
       + FK checks (use already opened prelocked tables)
       + purge threads (open the table, MDLock it, add it to tdc, close
         when not needed)
    - cache open tables in vc_templ
    - avoid unnecessary allocations, reuse table->record[0] and table->s->default_values
    - not needed in 5.7, because it overcalculates:
      + tell the server to calculate vcols for an on-going inline ADD INDEX
      + calculate vcols for correct error messages

  * update other engines (mroonga/tokudb) accordingly
2016-12-12 20:27:42 +01:00
Sergei Golubchik
b66976abb4 cleanup: InnoDB, various minor issues
* fix "unused pending_checkpoint_mutex_key" compiler warning
* clarify/simplify get_field_offset()
* typos, comments
* unused (forgotten) declaration of create_options_are_invalid()
* fix my_error(ER_WRONG_KEY_COLUMN) calls
* crash in row_upd_sec_index_entry()
* double if (ret != 0)
* don't duplucate PSI_INSTRUMENT_ME lines
* useless break; after return();
* remove unused xtradb-only "cursor_read_view" stuff
* code formatting
* simplify dropped column detection
* redundant assignment
2016-12-12 20:27:41 +01:00
Sergei Golubchik
2614a0ab0f extend prelocking to FK-accessed tables 2016-12-12 20:27:40 +01:00
Sergei Golubchik
f73bdb685d bugfix: UPDATE and virtual BLOBs
When updating a table with virtual BLOB columns, the following might happen:
- an old record is read from the table, it has no virtual blob values
- update_virtual_fields() is run, vcol blob gets its value into the
  record. But only a pointer to the value is in the table->record[0],
  the value is in Field_blob::value String (but it doesn't have to be!
  it can be in the record, if the column is just a copy of another
  columns: ... b VARCHAR, c BLOB AS (b) ...)
- store_record(table,record[1]), old record now is in record[1]
- fill_record() prepares new values in record[0], vcol blob is updated,
  new value replaces the old one in the Field_blob::value
- now both record[1] and record[0] have a pointer that points to the
  *new* vcol blob value. Or record[1] has a pointer to nowhere if
  Field_blob::value had to realloc.

To resolve this we unlink vcol blobs from the pointer to the
data (in the record[1]). Because the value is not *always* in
the Field_blob::value String, we need to remember what blobs
were unlinked. The orphan memory must be freed manually.

To complicate the matter, ha_update_row() is also used in
multi-update, in REPLACE, in INSERT ... ON DUPLICATE KEY UPDATE,
also on REPLACE ... SELECT, REPLACE DELAYED, and LOAD DATA REPLACE, etc
2016-12-12 20:27:38 +01:00
Sergei Golubchik
aebb1038aa bugfix: multi-UPDATE, vcols, const tables
multi-update was setting up read_set/vcol_set in
multi_update::initialize_tables() that is invoked after
the optimizer (JOIN::optimize_inner()). But some rows - if they're from
const tables - will be read already in the optimizer, and these rows
will not have all necessary column/vcol values.

* multi_update::initialize_tables() uses results from the optimizer
  and cannot be moved to be called earlier.
* multi_update::prepare() is called before the optimizer, but
  it cannot set up read_set/vcol_set, because the optimizer
  might reset them (see SELECT_LEX::update_used_tables()).

As a fix I've added a new method, select_result::prepare_to_read_rows(),
it's called from inside the optimizer just before make_join_statistics().
2016-12-12 20:27:38 +01:00
Sergei Golubchik
0e401bf7bf bugfix: move vcol calculations down into the handler
This fixes a bug where handler::read_range_first (for example)
needed to compare vcol values that were not calculated yet.

As a bonus it fixes few cases where vcols were calculated twice
2016-12-12 20:27:38 +01:00
Sergei Golubchik
b8f51c04d3 bugfix: update-behind-insert
sql_insert.cc calls handler->ha_update_row() for
REPLACE and INSERT... ON DUPLICATE KEY UPDATE
2016-12-12 20:27:37 +01:00
Sergei Golubchik
b38ff28ade bugfix: mark_columns_needed_for_update
cannot use TABLE:merge_keys for that, because Field::part_of_key
was designed to mark fields for KEY_READ, so a field is not a
"part of key", if only prefix of the field is.
2016-12-12 20:27:36 +01:00
Sergei Golubchik
d137b4dbba MDEV-5800 MyISAM support for indexed vcols
* don't issue an error for ER_KEY_BASED_ON_GENERATED_VIRTUAL_COLUMN
* support keyread on vcols
* callback into the server to compute vcol values from mi_check/mi_repair
* DMLs just work. Automatically.
2016-12-12 20:27:36 +01:00
Sergei Golubchik
a418c99200 gcol mysql-test suite from 5.7
update tests and results, fix bugs
2016-12-12 20:27:36 +01:00
Sergei Golubchik
4136968ca0 enable spatial indexes in innodb vcol tests 2016-12-12 20:27:35 +01:00
Sergei Golubchik
8b6c0542db bugfix: stored column depends on virtual depends on updated
TABLE::mark_virtual_col() was polluting table->vcol_set
and that confused the following mark_virtual_col()
2016-12-12 20:27:32 +01:00
Sergei Golubchik
7459f0c03a cleanup: don't update_virtual_fields from READ_RECORD
it was done only in some access methods, not in all,
so the caller had to update_virtual_fields anyway.
2016-12-12 20:27:31 +01:00
Sergei Golubchik
0bef3bb8d0 cleanup: remove Item::intro_version
and partition_info::set_show_version_string - they were
already broken and impossible to maintain
2016-12-12 20:27:25 +01:00
Sergei Golubchik
fd0044041b don't convert WEEK(x) to WEEK(x, @@default_week_format) 2016-12-12 20:27:24 +01:00
Sergei Golubchik
867809f23a bugfix: compile InnoDB w/o P_S 2016-12-12 20:27:23 +01:00
Sergei Golubchik
75925f8fc1 bugfix: Item_func_spatial_collection::print() 2016-12-12 20:27:23 +01:00
Sergei Golubchik
0852cf534a say MariaDB in InnoDB error messages, not MySQL 2016-12-12 20:27:21 +01:00
Marko Mäkelä
63152a52da Port the test innodb.innodb_misc1 from MySQL.
Adjust some results and error codes.
2016-12-09 12:00:19 +02:00
Jan Lindström
0009f4a441 Merge pull request #268 from grooverdan/10.2-MDEV-10651-enable-test-sys_vars.innodb_buffer_pool_dump_pct_basic
MDEV-10651: enable test sys_vars.innodb_buffer_pool_dump_pct_basic
2016-12-05 17:12:29 +02:00
Marko Mäkelä
cc85ba8f2e MDEV-11233 CREATE FULLTEXT INDEX with a token longer than 127 bytes
crashes server

This bug is the result of merging the Oracle MySQL follow-up fix
BUG#22963169 MYSQL CRASHES ON CREATE FULLTEXT INDEX
without merging the base bug fix:
Bug#79475 Insert a token of 84 4-bytes chars into fts index causes
server crash.

Unlike the above mentioned fixes in MySQL, our fix will not change
the storage format of fulltext indexes in InnoDB or XtraDB
when a character encoding with mbmaxlen=2 or mbmaxlen=3
and the length of a word is between 128 and 84*mbmaxlen bytes.
The Oracle fix would allocate 2 length bytes for these cases.

Compatibility with other MySQL and MariaDB releases is ensured by
persisting the used maximum length in the SYS_COLUMNS table in the
InnoDB data dictionary.

This fix also removes some unnecessary strcmp() calls when checking
for the legacy default collation my_charset_latin1
(my_charset_latin1.name=="latin1_swedish_ci").

fts_create_one_index_table(): Store the actual length in bytes.
This metadata will be written to the SYS_COLUMNS table.

fts_zip_initialize(): Initialize only the first byte of the buffer.
Actually the code should not even care about this first byte, because
the length is set as 0.

FTX_MAX_WORD_LEN: Define as HA_FT_MAXCHARLEN * 4 aka 336 bytes,
not as 254 bytes.

row_merge_create_fts_sort_index(): Set the actual maximum length of the
column in bytes, similar to fts_create_one_index_table().

row_merge_fts_doc_tokenize(): Remove the redundant parameter word_dtype.
Use the actual maximum length of the column. Calculate the extra_size
in the same way as row_merge_buf_encode() does.
2016-12-05 16:32:03 +02:00
Daniel Black
6924593ed8 MDEV-10651: enable test sys_vars.innodb_buffer_pool_dump_pct_basic
Since 10.2 (2e814d4702) this test
changed and the race condition of MDEV-10651 no longer forms part
of this test.

As such re-enable this test.

Also include save/restore of the default values of this variable.

Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
2016-12-05 10:42:42 +11:00
Marko Mäkelä
33ed16c750 MDEV-11236 Failing assertion: state == TRX_STATE_NOT_STARTED
trx_state_eq(): Add the parameter bool relaxed=false, to
allow trx->state==TRX_STATE_NOT_STARTED where a different
state is expected, if an error has been reported.

trx_release_savepoint_for_mysql(): Pass relaxed=true to
trx_state_eq(). That is, allow the transaction to be idle
when ROLLBACK TO SAVEPOINT is attempted after an error
has been reported to the client.
2016-12-02 16:44:53 +02:00
Sergei Golubchik
cb78555f30 update test results 2016-12-02 00:39:19 +01:00
Marko Mäkelä
0b66d3f70d MDEV-11426 Remove InnoDB INFORMATION_SCHEMA.FILES implementation
MySQL 5.7 introduced WL#7943: InnoDB: Implement Information_Schema.Files
to provide a long-term alternative for accessing tablespace metadata.
The INFORMATION_SCHEMA.INNODB_* views are considered internal interfaces
that are subject to change or removal between releases. So, users should
refer to I_S.FILES instead of I_S.INNODB_SYS_TABLESPACES to fetch metadata
about CREATE TABLESPACE.

Because MariaDB 10.2 does not support CREATE TABLESPACE or
CREATE TABLE…TABLESPACE for InnoDB, it does not make sense to support
I_S.FILES either. So, let MariaDB 10.2 omit the code that was added in
MySQL 5.7. After this change, I_S.FILES will report the empty result,
unless some other storage engine in MariaDB 10.2 implements the interface.
(The I_S.FILES interface was originally created for the NDB Cluster.)
2016-12-01 13:16:25 +02:00
Jan Lindström
943baa3ba8 MDEV-11168: InnoDB: Failing assertion: !other_lock || wsrep_thd_is_BF(lock->trx->mysql_thd, FALSE) || wsrep_thd_is_BF(other_lock->trx->mysql_thd, FALSE)
Problem was that we moved lock request to head of lock queue
    even when lock request has to wait.
2016-12-01 13:07:28 +02:00
Jan Lindström
dc9f919f27 MDEV-11005: Incorrect error message when using ONLINE alter table with GIS
Corrected error message when ONLINE alter table with GIS indexes is
used on InnoDB.
2016-12-01 06:42:59 +02:00
Alexey Botchkov
cc577f81b8 MDEV-11041 Innodb_gis/ tests taken from MySQL fail.
Some fixes made in innodb and item_create.cc.
        Adapted Innodb-GIS tests moved to MariaDB.
2016-11-29 16:25:22 +04:00
Alexander Barkov
9bfde897c3 MDEV-11360 Dynamic SQL: DEFAULT as a bind parameter
This patch adds DEFAULT as a possible dynamic SQL parameter, e.g.:
  EXECUTE IMMEDIATE 'INSERT INTO t1 (column) VALUES(?)' USING DEFAULT;
  EXECUTE IMMEDIATE 'UPDATE t1 SET column=?' USING DEFAULT;
and for similar PREPARE..EXECUTE queries.

This is done for symmetry with the STMT_INDICATOR_DEFAULT indicator in
the client-server PS protocol.

The changes include:
- Allowing DEFAULT as a possible option in execute USING clause (sql_yacc.yy)
- Adding "virtual bool Item::save_in_param(THD *thd, Item_param *param)",
  because "normal" items (that have real values) and Item_default_value
  have now different actions when assigning itself as an Item_param value.
- Fixing switch() statements in a few Item_param methods not to have "default",
  because it was easy to forget to add a new "case" when adding a new XXX_VALUE
  value into the enum Item_param::enum_item_param_state.
  This is important, as we'll be adding new values soon, e.g. for MDEV-11359.
  Removing "default" helped to find and report bugs MDEV-11361 and MDEV-11362,
  because DECIMAL_VALUE is obviously not properly handled in some cases.
2016-11-27 18:21:18 +04:00
Sergey Vojtovich
ded4cd12c3 Added missing INNODB_COMPRESSION_DEFAULT 2016-11-16 20:25:55 +04:00
Jan Lindström
0838fd00e7 MDEV-11185: innodb.innodb_trx_weight fails in buildbot
Problem was that test assumes locks to be granted on first-come-first-served (FCFS)
policy. However, in 10.2 we use by default Variance-Aware-Transaction-Scheduling
(VATS) algorithm. Test failure fixed by setting lock wait policy to FCFS.
2016-11-16 12:07:12 +02:00
Alexander Barkov
0259b3cbbe MDEV-11255 LDML: allow defining 2-level UCA collations 2016-11-08 20:57:19 +04:00
Nirbhay Choubey
491f42d50d Fix/disable some failing galera tests. 2016-11-07 11:53:59 -05:00
Nirbhay Choubey
9b6bd3f179 MDEV-11149: wsrep_replicate_mysaim: DML fails when binlog checksum enabled
During total-order replication, Query_log_event's checksum_alg
should be explicitly set to the current binlog_checksum as it
is not set for DML queries.
Note: wsrep_replicate_myisam enables replication of DMLs on
MyISAM tables using TOI.
2016-11-07 11:53:59 -05:00
Nirbhay Choubey
d5e6d8379d Update test results in galera, galera_3nodes suites. 2016-11-07 11:53:58 -05:00
Nirbhay Choubey
6bb6f30ff9 MDEV-9312: storage engine not enforced during galera cluster replication
Perform a post initialization of plugin-related variables
of wsrep threads after their global counterparts have been
initialized.
2016-11-07 11:53:56 -05:00
Sergey Vojtovich
458648e5da Fixed test suite name 2016-11-07 13:27:33 +04:00
Jan Lindström
ea24480c4e Merge pull request #255 from rasmushoj/MDEV-9820
MDEV-9820 Added server variable compression_default, which if 1/ON sets compres…
2016-11-07 10:25:49 +02:00
Elena Stepanova
f5719fcf8c Temporarily disable innodb.innodb_trx_weight test due to MDEV-11185 2016-11-07 03:20:04 +03:00
Rasmus Johansson
bba224dd53 Added server variable compression_default, which if 1/ON sets compression on for all new InnoDB/XtraDB tables by default by setting PAGE_COMPRESSED=1 2016-11-04 13:33:37 +02:00
Kristian Nielsen
7afcc7d1a2 Re-generate .rdiff file in attempt to fix a test failure.
The .rdiff applied ok locally with my copy of patch, but failed with
"misordered hunks" on a test host. Maybe that host has a more strict
version of `patch`.
2016-11-03 17:34:24 +01:00
Kristian Nielsen
b002509b67 MDEV-11065: Compressed binary log. Merge code into current 10.2.
Conflicts:
	sql/share/errmsg-utf8.txt
2016-11-03 14:48:51 +01:00
Sergey Vojtovich
d665e79c5b MDEV-7660 - MySQL WL#6671 "Improve scalability by not using thr_lock.c locks
for InnoDB tables"

Don't use thr_lock.c locks for InnoDB tables. Below is list of changes that
were needed to implement this:
- HANDLER OPEN acquireis MDL_SHARED_READ instead of MDL_SHARED
- HANDLER READ calls external_lock() even if SE is not going to be locked by
  THR_LOCK
- InnoDB lock wait timeouts are now honored which are much shorter by default
  than server lock wait timeouts (1 year vs 50 seconds)
- with @@autocommit= 1 LOCK TABLES disables autocommit implicitely, though
  user still sees @@autocommt= 1
- the above starts implicit transaction
- transactions started by LOCK TABLES are now rolled back on disconnect
  (previously everything was committed due to autocommit)
- transactions started by LOCK TABLES are now rolled back by ROLLBACK
  (previously everything was committed due to autocommit)
- it is now impossible to change BINLOG_FORMAT under LOCK TABLES (at least
  to statement) due to running transaction
- LOCK TABLES WRITE is additionally handled by MDL
- ...in contrast LOCK TABLES READ protection against DML is pure InnoDB
- combining transactional and non-transactional tables under LOCK TABLES
  may cause rolled back changes in transactional table and "committed"
  changes in non-transactional table
- user may disable innodb_table_locks, which will cause LOCK TABLES to be
  noop basically

Removed tests for BUG#45143 and BUG#55930 which cover InnoDB + THR_LOCK. To
operate properly these tests require code flow to go through THR_LOCK debug
sync points, which is not the case after this patch. These tests are removed
by WL#6671 as well. An alternative is to port them to different storage engine.
2016-11-03 16:35:15 +04:00
Kristian Nielsen
3c0ff6153f MDEV-11065: Compressed binary log
.result file update for --embedded test run.
2016-11-03 12:03:52 +01:00
Jan Lindström
0c15d1a6ff Set new scheduling algorithm VATS for lock waits as a default. 2016-10-25 12:21:54 +03:00