fts_get_next_doc_id(): Assign the first and subsequent FTS_DOC_ID
in the same way: by post-incrementing the cached value.
If there is a user-specified FTS_DOC_ID, do not touch the internal
sequence.
There are two bugs related to failed ADD INDEX and
the InnoDB table cache eviction.
dict_table_close(): Try dropping failed ADD INDEX when releasing
the last table handle, not when releasing the last-but-one.
dict_table_remove_from_cache_low(): Do not invoke
row_merge_drop_indexes() after freeing all index metadata.
Instead, directly invoke row_merge_drop_indexes_dict() to
remove the metadata from the persistent data dictionary
and to free the index pages.
This was the cause of the bug in CalculateArray
modified: storage/connect/jsonudf.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/value.cpp
- Avoid some compiler warnings
modified: storage/connect/mongo.cpp
modified: storage/connect/tabjmg.cpp
modified: storage/connect/tabtbl.cpp
modified: storage/connect/tabutil.cpp
modified: storage/connect/ha_connect.cc
- Include MONGO in all Java enabled distributions
Mongo will be enabled only for 10.2 and 10.3
modified: storage/connect/CMakeLists.txt
- Change JDBC_SUPPORT to JAVA_SUPPORT which also replaces MONGO_SUPPORT
MONGO_SUPPORT is now just used to enable the MONGO table type
modified: storage/connect/filter.cpp
modified: storage/connect/ha_connect.cc
modified: storage/connect/ha_connect.h
modified: storage/connect/mongo.cpp
modified: storage/connect/mycat.cc
modified: storage/connect/plgdbutl.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/tabjson.h
- Move MakeSelector function from FILTER to mongo.cpp
modified: storage/connect/filter.cpp
modified: storage/connect/filter.h
modified: storage/connect/cmgoconn.cpp
modified: storage/connect/jmgoconn.cpp
modified: storage/connect/mongo.cpp
- Do mongo_init only on first use of the MongoDB C Driver
This will permit to delay load the mongo lib on Windows
modified: storage/connect/cmgoconn.cpp
modified: storage/connect/cmgoconn.h
modified: storage/connect/ha_connect.cc
- Replace NEW_VAR by a test on MYSQL_VERSION_ID
modified: storage/connect/ha_connect.cc
- Suppress enable_mongo session variable
modified: storage/connect/ha_connect.cc
modified: storage/connect/mycat.cc
- Make some function headers identical in .h and .cc file
(replacing const char* by PCSZ)
modified: storage/connect/ha_connect.cc
modified: storage/connect/ha_connect.h
- Change a parameter type from uchar* to const uchar*
(for ScanRecord and CheckRecord)
modified: storage/connect/ha_connect.cc
modified: storage/connect/ha_connect.h
- Changes on LIKE and NOT LIKE does not fix a bug yet
modified: storage/connect/ha_connect.cc
- Suppress PIVOT_SUPPORT (PIVOT type is unconditionnal)
modified: storage/connect/ha_connect.cc
modified: storage/connect/mycat.cc
- Change the strz function from inline to static
modified: storage/connect/ha_connect.cc
modified: storage/connect/ha_connect.h
- export the JavaConn class and the MgoColumns and IsNum functions
modified: storage/connect/javaconn.h
modified: storage/connect/json.h
modified: storage/connect/mongo.h
- Fix MDEV-13924
modified: storage/connect/jdbconn.cpp
- Make a temporary fix for the compiler bug in CalculateArray
modified: storage/connect/jsonudf.cpp
modified: storage/connect/tabjson.cpp
- Typo
modified: storage/connect/jdbccat.h
modified: storage/connect/reldef.h
modified: storage/connect/tabext.h
modified: storage/connect/tabjmg.cpp
modified: storage/connect/tabxml.h
modified: storage/connect/valblk.h
modified: storage/connect/value.h
modified: storage/connect/xtable.h
- Fix a bug in MONGO tests by changing 'MONGO' to $TYPE
modified: storage/connect/mysql-test/connect/t/mongo_test.inc
- Record test results to reflect all changes
modified: storage/connect/mysql-test/connect/r/json_java_2.result
modified: storage/connect/mysql-test/connect/r/json_java_3.result
modified: storage/connect/mysql-test/connect/r/json_mongo_c.result
modified: storage/connect/mysql-test/connect/r/mongo_c.result
modified: storage/connect/mysql-test/connect/r/mongo_java_2.result
modified: storage/connect/mysql-test/connect/r/mongo_java_3.result
This was a false alarm in a debug check that was introduced in
commit 48192f963a which was a
10.2 code refactoring in preparation for
MDEV-11369 (instant ADD COLUMN) in 10.3.2. The code refactoring
only affected debug builds.
InnoDB B-tree record locks are only supposed to exist on leaf page
records. An assertion failed, because the debug function lock_validate()
was invoking lock_rec_block_validate() on a page for which there were
no locks set in the record lock bitmap. This could happen on a page split.
Especially when the index size grows from a single page to multiple pages,
the root page would transform from a leaf node into an internal node,
and its record lock bitmap would be emptied.
lock_validate(): Skip empty lock bitmaps.
Problem was that we could take page latches on different
order than wat is entitled with SX-lock. To follow the
latching order defined in WL#6326, acquire index->lock X-latch.
This entitles us to acquire page latches in any order for the index.
btr0btr.cc
Document latch rules before and after MariaDB 10.2.2
sync0rw.cc
Document latch compatibility rules better.
btr_defragment_merge_pages
Fix parameter value.
btr_defragment_thread
Acquire X-lock to dict_index_t::lock before restoring
cursor position and continuing defragmentation.
ha_innobase::optimize
Restore defragment feature.
Testing
Add GIS-index and FT-index to table being defragmented.
Defragmentation is not done to GIS-indexes and FT auxiliary
tables.
in 10.1 innodb was basically ignoring virtual columns. In particular,
information about them was not stored in system tables. To make 10.1
table usable in 10.2 it needs to be rebuilt to have virtual colunm
metadata properly recreated.
See also a followup:
MDEV-14046 Allow ALGORITHM=INPLACE for 10.1 tables that contain virtual columns
The fix 716f97e271
was inadvertently reverted in
commit 2e814d4702
"Merge InnoDB 5.7 from mysql-5.7.9".
Reapply the fix, because the test of the bug would fail
after merging MDEV-13838, which replaced an earlier incorrect
bug fix with a correct one.
Reverted incorrect changes done on MDEV-7367 and MDEV-9469. Fixes properly
also related bugs:
MDEV-13668: InnoDB unnecessarily rebuilds table when renaming a column and adding index
MDEV-9469: 'Incorrect key file' on ALTER TABLE
MDEV-9548: Alter table (renaming and adding index) fails with "Incorrect key file for table"
MDEV-10535: ALTER TABLE causes standalone/wsrep cluster crash
MDEV-13640: ALTER TABLE CHANGE and ADD INDEX on auto_increment column fails with "Incorrect key file for table..."
Root cause for all these bugs is the fact that MariaDB .frm file
can contain virtual columns but InnoDB dictionary does not and
previous fixes were incorrect or unnecessarily forced table
rebuilt. In index creation key_part->fieldnr can be bigger than
number of columns in InnoDB data dictionary. We need to skip not
stored fields when calculating correct column number for InnoDB
data dictionary.
dict_table_get_col_name_for_mysql
Remove
innobase_match_index_columns
Revert incorrect change done on MDEV-7367
innobase_need_rebuild
Remove unnecessary rebuild force when column is renamed.
innobase_create_index_field_def
Calculate InnoDB column number correctly and remove
unnecessary column name set.
innobase_create_index_def, innobase_create_key_defs
Remove unneeded fields parameter. Revert unneeded memset.
prepare_inplace_alter_table_dict
Remove unneeded col_names parameter
index_field_t
Remove unneeded col_name member.
row_merge_create_index
Remove unneeded col_names parameter and resolution.
Effected tests:
innodb-alter-table : Add test case for MDEV-13668
innodb-alter : Remove MDEV-13668, MDEV-9469 FIXMEs
and restore original tests
innodb-wl5980-alter : Remove MDEV-13668, MDEV-9469 FIXMEs
and restore original tests
Mariabackup 10.2.7 would delete the redo log files after a successful
--prepare operation. If the user is manually copying the prepared files
instead of using the --copy-back option, it could happen that some old
redo log file would be preserved in the restored location. These old
redo log files could cause corruption of the restored data files when
the server is started up.
We prevent this scenario by creating a "poisoned" redo log file
ib_logfile0 at the end of the --prepare step. The poisoning consists
of simply truncating the file to an empty file. InnoDB will refuse
to start up on an empty redo log file.
copy_back(): Delete all redo log files in the target if the source
file ib_logfile0 is empty. (Previously we did this if the source
file is missing.)
SRV_OPERATION_RESTORE_EXPORT: A new variant of SRV_OPERATION_RESTORE
when the --export option is specified. In this mode, we will keep
deleting all redo log files, instead of truncating the first one.
delete_log_files(): Add a parameter for the first file to delete,
to be passed as 0 or 1.
innobase_start_or_create_for_mysql(): In mariabackup --prepare,
tolerate an empty ib_logfile0 file. Otherwise, require the first
redo log file to be longer than 4 blocks (2048 bytes). Unless
--export was specified, truncate the first log file at the
end of --prepare.
Use GetFileInformationByHandleEx with FileAttributeTagInfo to query whether
the file is sparse. This saves 1 syscall, as GetFileInformationByHandle()
would additionally query volume info.
Try to fix fragmentation (unsparse files), for pre-existing
installations.
Unsparse the innodb file, when it needs to be extended, unless compression
is used. For Win7/2008R2 unsparse does not work (as documented in MSDN),
therefore for sparse files in older Windows, file extension will be done
via writing zeroes at the end of file.
The last parameter to this function is now,"bool is_sparse", like in 10.1
rather than the unused/useless "bool is_readonly", merged from MySQL 5.7
Like in 10.1, this function now supports sparse files, and efficient
platform specific mechanisms for file extension
os_file_set_size() is now consistenly used in all places where
innodb files are extended.
MariaDB 10.1 introduced non-indexed virtual columns for InnoDB tables.
When MySQL 5.7 introduced virtual columns in InnoDB tables, it also
introduced the table SYS_VIRTUAL that stores metadata on virtual
columns. This table does not initially exist in data files that were
imported from 10.1. So, we do not always have virtual column metadata
inside InnoDB.
dict_index_contains_col_or_prefix(): In the clustered index records,
all non-virtual columns are present and no virtual columns are present.
ha_innobase::build_template(): In the clustered index, do not
include virtual columns in the query template. The SQL layer is
supposed to compute the virtual column values when needed.
When btr_create() invokes btr_free_root() after running out of space,
fseg_create() would have acquired an SX-latch on the root page, not
an X-latch. Relax the debug assertion in btr_free_root() accordingly.
(In this case, SX-latch and X-latch are equivalent. During the CREATE
operation there should be MDL_EXCLUSIVE and dict_operation_lock X-latch
preventing concurrent access to the index. Normally the purpose of the
SX-latch is to allow concurrent reads of the root page while certain
fields in the root page are updated in place.)
fts_create_doc_id(): Remove.
row_mysql_convert_row_to_innobase(): Implement the logic of
fts_create_doc_id(). Reuse a buffer for the hidden FTS_DOC_ID.
row_get_prebuilt_insert_row(): Allocate a buffer for the hidden
FTS_DOC_ID at the end of prebuilt->ins_upd_rec_buff.
Some innobase/xtrabackup changes around from 10.1 are null merged
, in partucular using os_set_file_size to extend tablespaces in server
or mariabackup.
They require non-trivial amount of additional work in 10.2, due to
innobase differences between 10.1 and 10.2
In MariaDB Server 10.1, this problem manifests itself only as
a debug assertion failure in page_zip_decompress() when an insert
requires a page to be decompressed.
In MariaDB 10.1, the encryption of InnoDB data files repurposes the
previously unused field FILE_FLUSH_LSN for an encryption key version.
This field was only used in the first page of each file of the system
tablespace. For ROW_FORMAT=COMPRESSED tables, the field was always
written as 0 until encryption was implemented.
There is no bug in the encryption, because the buffer pool blocks will
not be written to files. Instead, copies of the blocks will be encrypted.
In these encrypted copies, the key version field will be updated before
the buffer is written to the file. The field in the buffer pool is
basically garbage that does not really matter.
Already in MariaDB 10.0, the memset() calls to reset this unused field
in buf_flush_update_zip_checksum() and buf_flush_write_block_low()
are unnecessary, because fsp_init_file_page_low() would guarantee that
the field is always 0 in the buffer pool (unless 10.1 encryption is
used).
Removing the unnecessary memset() calls makes page_zip_decompress()
happy and will prevent a SPATIAL INDEX corruption bug in
MariaDB Server 10.2. In MySQL 5.7.5, as part of WL#6968, the same
field was repurposed for an R-tree split sequence number (SSN) and
these memset() were removed. (Because of the repurposing, MariaDB
encryption is not available for tables that contain SPATIAL INDEX.)
fil_space_extend_must_retry(): If the table is being truncated,
do not call fil_flush_low(). The operation is covered by the
truncate log. File extension during TRUNCATE only occurs
if there are many indexes on the table. With smaller innodb_page_size,
the file extension occurs already with fewer indexes on the table.
The function dict_disable_redo_if_temporary() was supposed to
disable redo logging for temporary tables. It was invoked
unnecessarily for two read-only operations:
row_undo_search_clust_to_pcur() and
dict_stats_update_transient_for_index().
When a table is not temporary and not in the system tablespace,
the tablespace should be flagged for MLOG_FILE_NAME logging.
We do not need this overhead for temporary tables. Therefore,
either mtr_t::set_log_mode() or mtr_t::set_named_space() should
be invoked.
dict_table_t::is_temporary(): Determine if a table is temporary.
dict_table_is_temporary(): Redefined as a macro wrapper for
dict_table_t::is_temporary().
dict_disable_redo_if_temporary(): Remove.
The method handlerton::replace_native_transaction_in_thd was added
into MySQL 5.7 in WL#6860, Binlogging XA-prepared transaction.
In MariaDB we do not have that change yet.
The field dict_table_t::big_rows was only used for determining if
the adaptive hash index should be used when the internal InnoDB SQL
parser is used. That parser is only used for modifying the InnoDB
data dictionary, updating persistent tables, and for fulltext indexes.
