Commit graph

1371 commits

Author SHA1 Message Date
Sergei Golubchik
a7d6271cbf skip innodb.log_corruption test if no unzip executable is found
also move *.zip files from t/ to std_data/
2017-01-29 13:50:30 +01:00
Marko Mäkelä
2de0e42af5 Import and adjust the InnoDB redo log tests from MySQL 5.7. 2017-01-27 17:53:02 +02:00
Marko Mäkelä
0f34160d1d Clean up a few tests that kill the server.
As noted in MDEV-8841, any test that kills the server must issue
FLUSH TABLES, so that tables of crash-unsafe storage engines will
not be corrupted. Consistently issue this statement after any
call mtr.add_suppression() calls.

Also, do not invoke shutdown_server directly, but use helpers instead.
2017-01-27 17:07:45 +02:00
Jan Lindström
3dd6fca774 Fix test failure on innodb-trim.
Skip test if system can't do trim operation.
2017-01-27 14:18:41 +02:00
Marko Mäkelä
3271da11c4 MDEV-11814 Refuse innodb_read_only startup if crash recovery is needed
recv_scan_log_recs(): Remember if redo log apply is needed,
even if starting up in innodb_read_only mode.

recv_recovery_from_checkpoint_start_func(): Refuse
innodb_read_only startup if redo log apply is needed.
2017-01-27 13:01:56 +02:00
Marko Mäkelä
8daa87dc06 Fix a test. 2017-01-27 12:21:31 +02:00
Jan Lindström
4ee7174479 Fix test failure on innodb-trim.
Do wait only if innodb_num_page_compressed_trim_op shows that
we have succeed to do at least few trim operations (and
that will happen on insert if possible).
2017-01-27 11:26:28 +02:00
Jan Lindström
ebdf70db2a Fix test failure on innodb-wl5522
Added flush tables to avoid corruption of mtr/test_suppressions
table.
2017-01-27 09:13:48 +02:00
Jan Lindström
45f451c769 MDEV-11850: Can't create foreign key referencing a virtual column
Both dict_foreign_find_index and dict_foreign_qualify_index
did not consider virtual columns as possible foreign key
columns and there was assertion to disable virtual columns.

Fixed by also looking referencing and referenced column
from virtual columns if needed.
2017-01-26 08:05:23 +02:00
Marko Mäkelä
c6039a11c6 Fix a test. 2017-01-25 16:38:45 +02:00
Marko Mäkelä
e92ee13254 Import and adjust a test from MySQL 5.7. 2017-01-25 16:05:32 +02:00
Jan Lindström
6495806e59 MDEV-11254: innodb-use-trim has no effect in 10.2
Problem was that implementation merged from 10.1 was incompatible
with InnoDB 5.7.

buf0buf.cc: Add functions to return should we punch hole and
how big.

buf0flu.cc: Add written page to IORequest

fil0fil.cc: Remove unneeded status call and add test is
sparse files and punch hole supported by file system when
tablespace is created. Add call to get file system
block size. Used file node is added to IORequest. Added
functions to check is punch hole supported and setting
punch hole.

ha_innodb.cc: Remove unneeded status variables (trim512-32768)
and trim_op_saved. Deprecate innodb_use_trim and
set it ON by default. Add function to set innodb-use-trim
dynamically.

dberr.h: Add error code DB_IO_NO_PUNCH_HOLE
if punch hole operation fails.

fil0fil.h: Add punch_hole variable to fil_space_t and
block size to fil_node_t.

os0api.h: Header to helper functions on buf0buf.cc and
fil0fil.cc for os0file.h

os0file.h: Remove unneeded m_block_size from IORequest
and add bpage to IORequest to know actual size of
the block and m_fil_node to know tablespace file
system block size and does it support punch hole.

os0file.cc: Add function punch_hole() to IORequest
to do punch_hole operation,
get the file system block size and determine
does file system support sparse files (for punch hole).

page0size.h: remove implicit copy disable and
use this implicit copy to implement copy_from()
function.

buf0dblwr.cc, buf0flu.cc, buf0rea.cc, fil0fil.cc, fil0fil.h,
os0file.h, os0file.cc, log0log.cc, log0recv.cc:
Remove unneeded write_size parameter from fil_io
calls.

srv0mon.h, srv0srv.h, srv0mon.cc: Remove unneeded
trim512-trim32678 status variables. Removed
these from monitor tests.
2017-01-24 14:40:58 +02:00
Marko Mäkelä
b05bf8ff0f Merge 10.1 to 10.2.
Most notably, this includes MDEV-11623, which includes a fix and
an upgrade procedure for the InnoDB file format incompatibility
that is present in MariaDB Server 10.1.0 through 10.1.20.

In other words, this merge should address
MDEV-11202 InnoDB 10.1 -> 10.2 migration does not work
2017-01-19 12:06:13 +02:00
Jan Lindström
716b87845d Fix test failure on innodb-page_compression_snappy test.
Modified test data to be more easily compressed and
fixed mtr_suppression if compression fails.
2017-01-18 13:40:18 +02:00
Marko Mäkelä
08413254b7 Remove references to innodb_file_format.
innodb_file_format=Barracuda is the default in MariaDB 10.2.
Do not set it, because the option will be removed in MariaDB 10.3.

Also, do not set innodb_file_per_table=1 because it is the default.

Note that MDEV-11828 should fix the test innodb.innodb-64k
already in 10.1.
2017-01-18 08:43:11 +02:00
Marko Mäkelä
7cf97ed4ee MDEV-11816 Disallow CREATE TEMPORARY TABLE…ROW_FORMAT=COMPRESSED
MySQL 5.7 allows temporary tables to be created in ROW_FORMAT=COMPRESSED.
The usefulness of this is questionable. WL#7899 in MySQL 8.0.0
prevents the creation of such compressed tables, so that all InnoDB
temporary tables will be located inside the predefined
InnoDB temporary tablespace.

Pick up and adjust some tests from MySQL 5.7 and 8.0.

dict_tf_to_fsp_flags(): Remove the parameter is_temp.
fsp_flags_init(): Remove the parameter is_temporary.

row_mysql_drop_temp_tables(): Remove. There cannot be any temporary
tables in InnoDB. (This never removed #sql* tables in the datadir
which were created by DDL.)

dict_table_t::dir_path_of_temp_table: Remove.

create_table_info_t::m_temp_path: Remove.

create_table_info_t::create_options_are_invalid(): Do not allow
ROW_FORMAT=COMPRESSED or KEY_BLOCK_SIZE for temporary tables.

create_table_info_t::innobase_table_flags(): Do not unnecessarily
prevent CREATE TEMPORARY TABLE with SPATIAL INDEX.
(MySQL 5.7 does allow this.)

fil_space_belongs_in_lru(): The only FIL_TYPE_TEMPORARY tablespace
is never subjected to closing least-recently-used files.
2017-01-18 08:42:57 +02:00
Marko Mäkelä
494e4b99a4 Remove MYSQL_TABLESPACES.
MySQL 5.7 introduced partial support for user-created shared tablespaces
(for example, import and export are not supported).

MariaDB Server does not support tablespaces at this point of time.
Let us remove most InnoDB code and data structures that is related
to shared tablespaces.
2017-01-18 08:30:43 +02:00
Marko Mäkelä
7e3f3deb41 MDEV-11623 follow-up: Adjust tests.
innodb.doublewrite: Similar to what was done to innodb.101_compatibility,
add an explicit $_ parameter to the Perl unpack function.
Also, fix some diagnostic messages in the Perl code.

innodb.innodb-wl5522-debug: Adjust for the changed error codes and
messages on fault injection.
2017-01-16 11:23:12 +02:00
Elena Stepanova
3a91dec124 101_compatibility test fails on CentOS 5
Apparently, in older versions of Perl unpack does not have a logic
for using $_ as a default value for the second argument. Fixed by
specifying it explicitly
2017-01-16 02:36:31 +02:00
Marko Mäkelä
ab1e6fefd8 MDEV-11623 MariaDB 10.1 fails to start datadir created with
MariaDB 10.0/MySQL 5.6 using innodb-page-size!=16K

The storage format of FSP_SPACE_FLAGS was accidentally broken
already in MariaDB 10.1.0. This fix is bringing the format in
line with other MySQL and MariaDB release series.

Please refer to the comments that were added to fsp0fsp.h
for details.

This is an INCOMPATIBLE CHANGE that affects users of
page_compression and non-default innodb_page_size. Upgrading
to this release will correct the flags in the data files.
If you want to downgrade to earlier MariaDB 10.1.x, please refer
to the test innodb.101_compatibility how to reset the
FSP_SPACE_FLAGS in the files.

NOTE: MariaDB 10.1.0 to 10.1.20 can misinterpret
uncompressed data files with innodb_page_size=4k or 64k as
compressed innodb_page_size=16k files, and then probably fail
when trying to access the pages. See the comments in the
function fsp_flags_convert_from_101() for detailed analysis.

Move PAGE_COMPRESSION to FSP_SPACE_FLAGS bit position 16.
In this way, compressed innodb_page_size=16k tablespaces will not
be mistaken for uncompressed ones by MariaDB 10.1.0 to 10.1.20.

Derive PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR from the
dict_table_t::flags when the table is available, in
fil_space_for_table_exists_in_mem() or fil_open_single_table_tablespace().
During crash recovery, fil_load_single_table_tablespace() will use
innodb_compression_level for the PAGE_COMPRESSION_LEVEL.

FSP_FLAGS_MEM_MASK: A bitmap of the memory-only fil_space_t::flags
that are not to be written to FSP_SPACE_FLAGS. Currently, these will
include PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR.

Introduce the macro FSP_FLAGS_PAGE_SSIZE(). We only support
one innodb_page_size for the whole instance.

When creating a dummy tablespace for the redo log, use
fil_space_t::flags=0. The flags are never written to the redo log files.

Remove many FSP_FLAGS_SET_ macros.

dict_tf_verify_flags(): Remove. This is basically only duplicating
the logic of dict_tf_to_fsp_flags(), used in a debug assertion.

fil_space_t::mark: Remove. This flag was not used for anything.

fil_space_for_table_exists_in_mem(): Remove the unnecessary parameter
mark_space, and add a parameter for table flags. Check that
fil_space_t::flags match the table flags, and adjust the (memory-only)
flags based on the table flags.

fil_node_open_file(): Remove some redundant or unreachable conditions,
do not use stderr for output, and avoid unnecessary server aborts.

fil_user_tablespace_restore_page(): Convert the flags, so that the
correct page_size will be used when restoring a page from the
doublewrite buffer.

fil_space_get_page_compressed(), fsp_flags_is_page_compressed(): Remove.
It suffices to have fil_space_is_page_compressed().

FSP_FLAGS_WIDTH_DATA_DIR, FSP_FLAGS_WIDTH_PAGE_COMPRESSION_LEVEL,
FSP_FLAGS_WIDTH_ATOMIC_WRITES: Remove, because these flags do not
exist in the FSP_SPACE_FLAGS but only in memory.

fsp_flags_try_adjust(): New function, to adjust the FSP_SPACE_FLAGS
in page 0. Called by fil_open_single_table_tablespace(),
fil_space_for_table_exists_in_mem(), innobase_start_or_create_for_mysql()
except if --innodb-read-only is active.

fsp_flags_is_valid(ulint): Reimplement from the scratch, with
accurate comments. Do not display any details of detected
inconsistencies, because the output could be confusing when
dealing with MariaDB 10.1.x data files.

fsp_flags_convert_from_101(ulint): Convert flags from buggy
MariaDB 10.1.x format, or return ULINT_UNDEFINED if the flags
cannot be in MariaDB 10.1.x format.

fsp_flags_match(): Check the flags when probing files.
Implemented based on fsp_flags_is_valid()
and fsp_flags_convert_from_101().

dict_check_tablespaces_and_store_max_id(): Do not access the
page after committing the mini-transaction.

IMPORT TABLESPACE fixes:

AbstractCallback::init(): Convert the flags.

FetchIndexRootPages::operator(): Check that the tablespace flags match the
table flags. Do not attempt to convert tablespace flags to table flags,
because the conversion would necessarily be lossy.

PageConverter::update_header(): Write back the correct flags.
This takes care of the flags in IMPORT TABLESPACE.
2017-01-15 19:05:50 +02:00
Marko Mäkelä
a9d00db155 MDEV-11799 InnoDB can abort if the doublewrite buffer
contains a bad and a good copy

Clean up the InnoDB doublewrite buffer code.

buf_dblwr_init_or_load_pages(): Do not add empty pages to the buffer.

buf_dblwr_process(): Do consider changes to pages that are all zero.
Do not abort when finding a corrupted copy of a page in the doublewrite
buffer, because there could be multiple copies in the doublewrite buffer,
and only one of them needs to be good.
2017-01-15 18:56:56 +02:00
Marko Mäkelä
7c81f15ec3 Merge 10.1 into bb-10.2-mdev-11782 2017-01-12 13:00:11 +02:00
Marko Mäkelä
2dc5d8bb7e Improve an MDEV-9011 test of innodb_encrypt_log.
Test crash recovery from an encrypted redo log with innodb_encrypt_log=0.
Previously, we did a clean shutdown, so only the log checkpoint
information would have been read from the redo log. With this change,
we will be reading and applying encrypted redo log records.

include/start_mysqld.inc: Observe $restart_parameters.

encryption.innodb-log-encrypt: Remove some unnecessary statements,
and instead of restarting the server and concurrently accessing
the files while the server is running, kill the server, check the
files, and finally start up the server.

innodb.log_data_file_size: Use start_mysqld.inc with $restart_parameters.
2017-01-12 12:33:46 +02:00
Marko Mäkelä
3d46768da2 Merge 10.1 into 10.2 2017-01-09 09:47:12 +02:00
Marko Mäkelä
59ea6456c3 Minor cleanup of innodb.innodb-change-buffer-recovery
This should be a non-functional change. I was unable to repeat
MDEV-11626 innodb.innodb-change-buffer-recovery fails for xtradb
and cannot determine the reason for the failure without having access
to the files.

The repeatability of MDEV-11626 should not be affected by these changes.
2017-01-09 09:28:00 +02:00
Marko Mäkelä
9ebd767331 Merge 10.1 into 10.2 2016-12-30 13:48:22 +02:00
Marko Mäkelä
8451e09073 MDEV-11556 InnoDB redo log apply fails to adjust data file sizes
fil_space_t::recv_size: New member: recovered tablespace size in pages;
0 if no size change was read from the redo log,
or if the size change was implemented.

fil_space_set_recv_size(): New function for setting space->recv_size.

innodb_data_file_size_debug: A debug parameter for setting the system
tablespace size in recovery even when the redo log does not contain
any size changes. It is hard to write a small test case that would
cause the system tablespace to be extended at the critical moment.

recv_parse_log_rec(): Note those tablespaces whose size is being changed
by the redo log, by invoking fil_space_set_recv_size().

innobase_init(): Correct an error message, and do not require a larger
innodb_buffer_pool_size when starting up with a smaller innodb_page_size.

innobase_start_or_create_for_mysql(): Allow startup with any initial
size of the ibdata1 file if the autoextend attribute is set. Require
the minimum size of fixed-size system tablespaces to be 640 pages,
not 10 megabytes. Implement innodb_data_file_size_debug.

open_or_create_data_files(): Round the system tablespace size down
to pages, not to full megabytes, (Our test truncates the system
tablespace to more than 800 pages with innodb_page_size=4k.
InnoDB should not imagine that it was truncated to 768 pages
and then overwrite good pages in the tablespace.)

fil_flush_low(): Refactored from fil_flush().

fil_space_extend_must_retry(): Refactored from
fil_extend_space_to_desired_size().

fil_mutex_enter_and_prepare_for_io(): Extend the tablespace if
fil_space_set_recv_size() was called.

The test case has been successfully run with all the
innodb_page_size values 4k, 8k, 16k, 32k, 64k.
2016-12-30 09:52:24 +02:00
Marko Mäkelä
f493e395b0 Make the test work with any innodb_page_size. 2016-12-30 09:51:11 +02:00
Marko Mäkelä
341c375d4b Merge 10.1 into 10.2 2016-12-30 08:53:54 +02:00
Marko Mäkelä
7bcae22bf1 Merge branch 'bb-10.2-mdev-6076' into 10.2 2016-12-29 15:05:04 +02:00
Sergei Golubchik
4a5d25c338 Merge branch '10.1' into 10.2 2016-12-29 13:23:18 +01:00
Marko Mäkelä
08f79bdeda MDEV-11635 innodb.innodb_mysql test hangs
Copy and adapt the test from MySQL 5.7.17.
2016-12-22 16:48:49 +02:00
Sergey Vojtovich
0c3791caae MDEV-11227 - mysqlimport -l doesn't issue UNLOCK TABLES
Merged fix for innodb_mysql from 5.7.
2016-12-22 14:02:27 +04:00
Sergey Vojtovich
8774a02364 MDEV-11227 - mysqlimport -l doesn't issue UNLOCK TABLES
Implementation of MDEV-7660 introduced unwanted incompatible change:
modifications under LOCK TABLES with autocommit enabled are rolled back on
disconnect. Previously everything was committed, because LOCK TABLES didn't
adjust autocommit setting.

This patch restores original behavior by reverting some changes done in
MDEV-7660:
- sql/sql_parse.cc: do not reset autocommit on LOCK TABLES
- sql/sql_base.cc: do not set autocommit on UNLOCK TABLES
- test cases: main.lock_tables_lost_commit, main.partition_explicit_prune,
  rpl.rpl_switch_stm_row_mixed, tokudb.nested_txn_implicit_commit,
  tokudb_bugs.db806

But it makes InnoDB tables under LOCK TABLES ... READ [LOCAL] not protected
against DML. To restore protection some changes from WL#6671 were merged,
specifically MDL_SHARED_READ_ONLY and test cases.

WL#6671 merge highlights:
- Not all tests merged.
- In MySQL LOCK TABLES ... READ acquires MDL_SHARED_READ_ONLY for all engines,
  in MariaDB MDL_SHARED_READ is always acquired first and then upgraded to
  MDL_SHARED_READ_ONLY for InnoDB only.
- The above allows us to omit MDL_SHARED_WRITE_LOW_PRIO implementation in
  MariaDB, which is rather useless with InnoDB. In MySQL it is needed to
  preserve locking behavior between low priority writes and LOCK TABLES ... READ
  for non-InnoDB engines (covered by sys_vars.sql_low_priority_updates_func).
- Omitted HA_NO_READ_LOCAL_LOCK, we rely on lock_count() instead.
- Omitted "piglets": in MariaDB stream of DML against InnoDB table may lead to
  concurrent LOCK TABLES ... READ starvation.
- HANDLER ... OPEN acquires MDL_SHARED_READ instead of MDL_SHARED in MariaDB.
- Omitted SNRW->X MDL lock upgrade for IMPORT/DISCARD TABLESPAECE under LOCK
  TABLES.
- Omitted strong locks for views, triggers and SP under LOCK TABLES.
- Omitted IX schema lock for LOCK TABLES READ.
- Omitted deadlock weight juggling for LOCK TABLES.

Full WL#6671 merge status:
- innodb.innodb-lock: fully merged
- main.alter_table: not merged due to different HANDLER solution
- main.debug_sync: fully merged
- main.handler_innodb: not merged due to different HANDLER solution
- main.handler_myisam: not merged due to different HANDLER solution
- main.innodb_mysql_lock: fully merged
- main.insert_notembedded: fully merged
- main.lock: not merged (due to no strong locks for views)
- main.lock_multi: not merged
- main.lock_sync: fully merged (partially in MDEV-7660)
- main.mdl_sync: not merged
- main.partition_debug_sync: not merged due to different HANDLER solution
- main.status: fully merged
- main.view: fully merged
- perfschema.mdl_func: not merged (no such test in MariaDB)
- perfschema.table_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
- perfschema.table_lock_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
- sys_vars.sql_low_priority_updates_func: not merged
- include/thr_rwlock.h: not merged, rw_pr_lock_assert_write_owner and
  rw_pr_lock_assert_not_write_owner are macros in MariaDB
- sql/handler.h: not merged (HA_NO_READ_LOCAL_LOCK)
- sql/mdl.cc: partially merged (MDL_SHARED_READ_ONLY only)
- sql/mdl.h: partially merged (MDL_SHARED_READ_ONLY only)
- sql/lock.cc: fully merged
- sql/sp_head.cc: not merged
- sql/sp_head.h: not merged
- sql/sql_base.cc: partially merged (MDL_SHARED_READ_ONLY only)
- sql/sql_base.h: not merged
- sql/sql_class.cc: fully merged
- sql/sql_class.h: fully merged
- sql/sql_handler.cc: merged partially (different solution in MariaDB)
- sql/sql_parse.cc: partially merged, mostly omitted low priority write part
- sql/sql_reload.cc: not merged comment change
- sql/sql_table.cc: not merged SNRW->X upgrade for IMPORT/DISCARD TABLESPACE
- sql/sql_view.cc: not merged
- sql/sql_yacc.yy: not merged (MDL_SHARED_WRITE_LOW_PRIO, MDL_SHARED_READ_ONLY)
- sql/table.cc: not merged (MDL_SHARED_WRITE_LOW_PRIO)
- sql/table.h: not merged (MDL_SHARED_WRITE_LOW_PRIO)
- sql/trigger.cc: not merged
- storage/innobase/handler/ha_innodb.cc: merged store_lock()/lock_count()
  changes (in MDEV-7660), didn't merge HA_NO_READ_LOCAL_LOCK
- storage/innobase/handler/ha_innodb.h: fully merged in MDEV-7660
- storage/myisammrg/ha_myisammrg.cc: not merged comment change
- storage/perfschema/table_helper.cc: not merged (no MDL support in MariaDB PFS)
- unittest/gunit/mdl-t.cc: not merged
- unittest/gunit/mdl_sync-t.cc: not merged

MariaDB specific changes:
- handler.heap: different HANDLER solution, MDEV-7660
- handler.innodb: different HANDLER solution, MDEV-7660
- handler.interface: different HANDLER solution, MDEV-7660
- handler.myisam: different HANDLER solution, MDEV-7660
- main.mdl_sync: MDEV-7660 specific changes
- main.partition_debug_sync: removed test due to different HANDLER solution,
  MDEV-7660
- main.truncate_coverage: removed test due to different HANDLER solution,
  MDEV-7660
- mysql-test/include/mtr_warnings.sql: additional cleanup, MDEV-7660
- mysql-test/lib/v1/mtr_report.pl: additional cleanup, MDEV-7660
- plugin/metadata_lock_info/metadata_lock_info.cc: not in MySQL
- sql/sql_handler.cc: MariaDB specific fix for mysql_ha_read(), MDEV-7660
2016-12-21 13:18:45 +04:00
Marko Mäkelä
195241e125 Port the test innodb.doublewrite from MySQL 5.7. 2016-12-20 15:03:56 +02:00
Marko Mäkelä
c64edc6b83 MDEV-6076: Preserve PAGE_ROOT_AUTO_INC when emptying pages.
Thanks to Zhangyuan from Alibaba for pointing out this bug.

btr_page_empty(): When a clustered index root page is emptied,
preserve PAGE_ROOT_AUTO_INC. This would occur during a page split.

page_create_empty(): Preserve PAGE_ROOT_AUTO_INC when a clustered
index root page becomes empty. Use a faster method for writing
the field.

page_zip_copy_recs(): Reset PAGE_MAX_TRX_ID when copying
clustered index pages. We must clear the field when the root page
was a leaf page and it is being split, so that PAGE_MAX_TRX_ID
will continue to be 0 in clustered index non-root pages.

page_create_zip(): Add debug assertions for validating
PAGE_MAX_TRX_ID and PAGE_ROOT_AUTO_INC.
2016-12-16 10:26:41 +02:00
Marko Mäkelä
cb0ce5c2e9 MDEV-6076: Optimize the test.
Remove unnecessary restarts by testing multiple tables across a restart.
This change almost halves the execution time.
Some further restarts could be removed with additional effort.
2016-12-16 10:24:54 +02:00
Marko Mäkelä
8777458a6e MDEV-6076 Persistent AUTO_INCREMENT for InnoDB
This should be functionally equivalent to WL#6204 in MySQL 8.0.0, with
the notable difference that the file format changes are limited to
repurposing a previously unused data field in B-tree pages.

For persistent InnoDB tables, write the last used AUTO_INCREMENT
value to the root page of the clustered index, in the previously
unused (0) PAGE_MAX_TRX_ID field, now aliased as PAGE_ROOT_AUTO_INC.
Unlike some other previously unused InnoDB data fields, this one was
actually always zero-initialized, at least since MySQL 3.23.49.

The writes to PAGE_ROOT_AUTO_INC are protected by SX or X latch on the
root page. The SX latch will allow concurrent read access to the root
page. (The field PAGE_ROOT_AUTO_INC will only be read on the
first-time call to ha_innobase::open() from the SQL layer. The
PAGE_ROOT_AUTO_INC can only be updated when executing SQL, so
read/write races are not possible.)

During INSERT, the PAGE_ROOT_AUTO_INC is updated by the low-level
function btr_cur_search_to_nth_level(), adding no extra page
access. [Adaptive hash index lookup will be disabled during INSERT.]

If some rare UPDATE modifies an AUTO_INCREMENT column, the
PAGE_ROOT_AUTO_INC will be adjusted in a separate mini-transaction in
ha_innobase::update_row().

When a page is reorganized, we have to preserve the PAGE_ROOT_AUTO_INC
field.

During ALTER TABLE, the initial AUTO_INCREMENT value will be copied
from the table. ALGORITHM=COPY and online log apply in LOCK=NONE will
update PAGE_ROOT_AUTO_INC in real time.

innodb_col_no(): Determine the dict_table_t::cols[] element index
corresponding to a Field of a non-virtual column.
(The MySQL 5.7 implementation of virtual columns breaks the 1:1
relationship between Field::field_index and dict_table_t::cols[].
Virtual columns are omitted from dict_table_t::cols[]. Therefore,
we must translate the field_index of AUTO_INCREMENT columns into
an index of dict_table_t::cols[].)

Upgrade from old data files:

By default, the AUTO_INCREMENT sequence in old data files would appear
to be reset, because PAGE_MAX_TRX_ID or PAGE_ROOT_AUTO_INC would contain
the value 0 in each clustered index page. In new data files,
PAGE_ROOT_AUTO_INC can only be 0 if the table is empty or does not contain
any AUTO_INCREMENT column.

For backward compatibility, we use the old method of
SELECT MAX(auto_increment_column) for initializing the sequence.

btr_read_autoinc(): Read the AUTO_INCREMENT sequence from a new-format
data file.

btr_read_autoinc_with_fallback(): A variant of btr_read_autoinc()
that will resort to reading MAX(auto_increment_column) for data files
that did not use AUTO_INCREMENT yet. It was manually tested that during
the execution of innodb.autoinc_persist the compatibility logic is
not activated (for new files, PAGE_ROOT_AUTO_INC is never 0 in nonempty
clustered index root pages).

initialize_auto_increment(): Replaces
ha_innobase::innobase_initialize_autoinc(). This initializes
the AUTO_INCREMENT metadata. Only called from ha_innobase::open().

ha_innobase::info_low(): Do not try to lazily initialize
dict_table_t::autoinc. It must already have been initialized by
ha_innobase::open() or ha_innobase::create().

Note: The adjustments to class ha_innopart were not tested, because
the source code (native InnoDB partitioning) is not being compiled.
2016-12-16 09:19:19 +02:00
Marko Mäkelä
65b4d7457e Merge the test innodb.innodb_misc1 into innodb.innodb. 2016-12-13 11:52:23 +02:00
Sergei Golubchik
180065ebb0 Item::print(): remove redundant parentheses
by introducing new Item::precedence() method and using it
to decide whether parentheses are required
2016-12-12 20:44:41 +01:00
Sergei Golubchik
1db438c833 MDEV-11066 use MySQL terminology for "virtual columns" 2016-12-12 20:35:51 +01:00
Sergei Golubchik
a411d7f4f6 store/show vcols as item->print()
otherwise we'd need to store sql_mode *per vcol*
(consider CREATE INDEX...) and how SHOW CREATE TABLE would
support that?

Additionally, get rid of vcol::expr_str, just to make sure
the string is always generated and never leaked in the
original form.
2016-12-12 20:35:41 +01:00
Sergei Golubchik
1cae1af6f9 MDEV-5800 InnoDB support for indexed vcols
* remove old 5.2+ InnoDB support for virtual columns
  * enable corresponding parts of the innodb-5.7 sources
  * copy corresponding test cases from 5.7
  * copy detailed Alter_inplace_info::HA_ALTER_FLAGS flags from 5.7
     - and more detailed detection of changes in fill_alter_inplace_info()
  * more "innodb compatibility hooks" in sql_class.cc to
     - create/destroy/reset a THD (used by background purge threads)
     - find a prelocked table by name
     - open a table (from a background purge thread)

  * different from 5.7:
    - new service thread "thd_destructor_proxy" to make sure all THDs are
      destroyed at the correct point in time during the server shutdown
    - proper opening/closing of tables for vcol evaluations in
       + FK checks (use already opened prelocked tables)
       + purge threads (open the table, MDLock it, add it to tdc, close
         when not needed)
    - cache open tables in vc_templ
    - avoid unnecessary allocations, reuse table->record[0] and table->s->default_values
    - not needed in 5.7, because it overcalculates:
      + tell the server to calculate vcols for an on-going inline ADD INDEX
      + calculate vcols for correct error messages

  * update other engines (mroonga/tokudb) accordingly
2016-12-12 20:27:42 +01:00
Sergei Golubchik
2614a0ab0f extend prelocking to FK-accessed tables 2016-12-12 20:27:40 +01:00
Sergei Golubchik
0852cf534a say MariaDB in InnoDB error messages, not MySQL 2016-12-12 20:27:21 +01:00
Sergei Golubchik
2f20d297f8 Merge branch '10.0' into 10.1 2016-12-11 09:53:42 +01:00
Marko Mäkelä
63152a52da Port the test innodb.innodb_misc1 from MySQL.
Adjust some results and error codes.
2016-12-09 12:00:19 +02:00
Sergei Golubchik
7f2fd34500 MDEV-11231 Server crashes in check_duplicate_key on CREATE TABLE ... SELECT
be consistent and don't include the table name into the error message,
no other CREATE TABLE error does it.

(the crash happened, because thd->lex->query_tables was NULL)
2016-12-04 01:59:35 +01:00
Sergei Golubchik
c5ef621599 Merge branch 'merge/merge-tokudb-5.6' into 10.0 2016-12-04 01:59:08 +01:00
Sergei Golubchik
e4a0d75a0b import a test case from percona-server-5.6.34-79.1 2016-12-04 01:35:57 +01:00