The problem was that my_hash_sort didn't properly delete end-space characters properly, so strings that should compare
identically was seen as different strings. (Space was handled correctly, but not NBSP)
This caused duplicate key errors when a heap table was converted to Aria as part of overflow in group by.
Fixed by removing all characters that compares as end space when creating a hash.
Other things:
- Fixed that --sorted_results also works for errors in mysqltest.
- Speed up hash by not comparing strings that has different hash.
- Speed up many my_hash_sort functions by using registers to calculate hash instead of pointers.
This was previously done for some functions, but not for all.
- Made a macro of the hash function, to simplify code and to be able to experiment with new hash functions.
client/mysqltest.cc:
Fixed that --sorted_results also works for error messages.
mysql-test/r/ctype_partitions.result:
New test to ensure that partitions on hash works
mysql-test/suite/multi_source/gtid.result:
Updated result
mysql-test/suite/multi_source/gtid.test:
Test that --sorted_result works for error messages
mysql-test/suite/multi_source/gtid_ignore_duplicates.result:
Updated result
mysql-test/suite/multi_source/gtid_ignore_duplicates.test:
Updated result
mysql-test/suite/multi_source/load_data.result:
Updated result
mysql-test/suite/multi_source/load_data.test:
Updated result
mysql-test/t/ctype_partitions.test:
New test to ensure that partitions on hash works
storage/heap/hp_write.c:
Speed up hash by not comparing strings that has different hash.
storage/maria/ma_check.c:
Extra debug
strings/ctype-bin.c:
Use macro for hash function
strings/ctype-latin1.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-mb.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-simple.c:
Use macro for hash function
Use same variable names as in other my_hash_sort functions.
Update my_hash_sort_simple() to properly remove end space (patch by Bar)
strings/ctype-uca.c:
Ignore duplicated space inside strings and end space in my_hash_sort_uca(). This fixed MDEV-6255
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-ucs2.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/ctype-utf8.c:
Use macro for hash function
Use registers to calculate hash (speedup)
strings/strings_def.h:
Made a macro of the hash function, to simplify code and to be able to experiment with new hash functions.
file trx0trx.ic line 60
Problem was that trx might have not been started when we enter release
savepoint, this can happen when trx with savepoint is already aborted
or when we try to release non-existing savepoint.
* handler::get_auto_increment() was not expecting any errors from the storage engine.
That was wrong, errors could happen.
* ha_partition::get_auto_increment() was doing index lookups in partition under a mutex.
This was redundant (engine transaction isolation was covering that anyway)
and harmful (causing deadlocks).
that uninitialised characters be written in op file.
modified:
storage/connect/tabdos.cpp
- Typo
modified:
storage/connect/filamap.cpp
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/valblk.cpp
storage/connect/xindex.cpp
MDEV-6483 - Deadlock around rw_lock_debug_mutex on PPC64
This problem affects only debug builds on PPC64.
There are at least two race conditions around
rw_lock_debug_mutex_enter and rw_lock_debug_mutex_exit:
- rw_lock_debug_waiters was loaded/stored without setting
appropriate locks/memory barriers.
- there is a gap between calls to os_event_reset() and
os_event_wait() and in such case we're supposed to pass
return value of the former to the latter.
Fixed by replacing self-cooked spinlocks with system mutexes.
These days system mutexes offer much better performance. OTOH
performance is not that critical for debug builds.
MDEV-6450 - MariaDB crash on Power8 when built with advance tool
chain
InnoDB mutex_exit() function calls __sync_test_and_set() to release
the lock. According to manual this function is supposed to create
"acquire" memory barrier whereas in fact we need "release" memory
barrier at mutex_exit().
The problem isn't repeatable with gcc because it creates
"acquire-release" memory barrier for __sync_test_and_set().
ATC creates just "acquire" barrier.
Fixed by creating proper barrier at mutex_exit() by using
__sync_lock_release() instead of __sync_test_and_set().
tables. This caused a CR character to be left in the line and in
particular caused the updelx test to fail on Windows.
modified:
storage/connect/filamtxt.cpp
while returning an error on Linux. Now accepted on linux.
modified:
storage/connect/maputil.cpp
- Fix a BUG in the XHUGE class. lseek64 was wrongly regarded as in error
when returning 0 instead of -1. This produced wrong index files.
modified:
storage/connect/filamfix.cpp
storage/connect/maputil.cpp
storage/connect/xindex.cpp
- Fix length mismatch (tab instead of blanks?)
modified:
storage/connect/mysql-test/connect/r/updelx.result
(while doing indexed UPDATE/DELETE)
modified:
storage/connect/array.cpp
storage/connect/filamtxt.cpp
- Force unix like line endings
modified:
storage/connect/tabvct.h
Analysis: When database is migrated from 5.5 or earlier and
database needs crash recovery, there is possibility that
SYS_DATAFILES system table does not exists, but
crash recovery in function dict_check_tablespaces_and_store_max_id()
assumes that SYS_DATAFILES exists.
Fix: If SYS_DATAFILES does not exists, create it before
we end up to function dict_check_tablespaces_and_store_max_id()
on crash recovery.
it belongs to each tables and the catalog being share between several
instances of CONNECT, when a query implied several tables belonging to
different databases, some where pointing on the wrong database. This
fix bugs occuring in queries such as:
INSERT into db1.t1 select * from db2.t2;
Where the t1 data file was made in db2.
modified:
storage/connect/catalog.h
storage/connect/connect.cc
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/mycat.cc
storage/connect/mycat.h
storage/connect/plgdbsem.h
storage/connect/plgdbutl.cpp
storage/connect/reldef.cpp
storage/connect/reldef.h
storage/connect/tabfix.h
storage/connect/tabfmt.cpp
storage/connect/tabfmt.h
storage/connect/tabmul.cpp
a temporary file should be used for UPDATE/DELETE of file tables.
Also use the "sorted" argument of index_init to help decide if
sorting of positions must be done.
modified:
storage/connect/checklvl.h
storage/connect/connect.cc
storage/connect/connect.h
storage/connect/filamdbf.cpp
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/ha_connect.cc
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/plgdbsem.h
storage/connect/plgdbutl.cpp
storage/connect/reldef.cpp
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabfix.cpp
storage/connect/tabfmt.cpp
storage/connect/tabvct.cpp
storage/connect/tabvct.h
storage/connect/xindex.cpp
- Fix a bug in TDBASE::ColDB that caused some special columns not to
be found in the column list and reallocated without their Value
causing a crash of some queries.
modified:
storage/connect/table.cpp
- Fix a bug causing RestoreNrec to be called before closing a table
causing a wrong value given to Spos
modified:
storage/connect/tabdos.cpp
storage/connect/xindex.cpp
- Add a new CONNECT global variable connect_exact_info. Set to ON, it
tells CONNECT to return exact record numbers on info queries. If OFF
it just gives an estimate. In version 10.0.13 this was unconditionally
ON and caused info queries on remote tables to be extremely long and
was the subject of MDEV-6612.
modified:
storage/connect/ha_connect.cc
storage/connect/tabdos.cpp
storage/connect/tabmysql.cpp
storage/connect/tabodbc.cpp
Part of this work is based on Stewart Smitch's memory barrier and lower priori
patches for power8.
- Added memory syncronization for innodb & xtradb for power8.
- Added HAVE_WINDOWS_MM_FENCE to CMakeList.txt
- Added os_isync to fix a syncronization problem on power
- Added log_get_lsn_nowait which is now used srv_error_monitor_thread to ensur
if log mutex is locked.
All changes done both for InnoDB and Xtradb
them sorted by file position. Firstly a new value is stored in indexes
to know if they are sorted, preventing to do the sorting when it is not
needed. Secondly, almost all in now done in connect instead of being
done by the different file access method classes. This pepares the future
use of temporary files for all table types and also fix the bug that was
occuring when partially using a multi-column index because of false MRR
like call of position followed by unsorted rnd_pos no more using indexing.
modified:
storage/connect/connect.cc
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabfix.h
storage/connect/tabfmt.cpp
storage/connect/tabfmt.h
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
config.h.cmake:
define NOMINMAX, otherwise Windows system headers define min() and max() macros
sql/slave.cc:
mi->report() has one more argument in MariaDB
storage/xtradb/buf/buf0flu.cc:
xtradb fixes for windows, again
have been created by the last addition of new CONNECT features.
The version previous to this one is a preliminary test version and
should not be distributed.
- Handle indexed UPDATE/DELETE. Previously this was just tested and
an error message send when it could not be done. Now CONNECT can
do it in all the cases. It is done by a MRR like tchnique by making
a list of all update or delete to do, sort them, then execute them.
modified:
storage/connect/array.cpp
storage/connect/array.h
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/global.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Differenciate Cardinality that returns a true or estimated table size
and GetMaxSize that return a value equal or greater than the table
row number. This fixes the errors of non matching opt files.
modified:
storage/connect/connect.cc
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabfix.cpp
storage/connect/table.cpp
storage/connect/tabmac.h
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
storage/connect/tabpivot.h
storage/connect/tabtbl.cpp
storage/connect/tabtbl.h
storage/connect/tabutil.cpp
storage/connect/tabutil.h
storage/connect/tabwmi.h
storage/connect/xtable.h
- Fix some errors and issues when making index and opt files.
Erase opt and index files for void tables.
Fix wrong calculation of Block and Last in MakeBlockValues.
Invalidate indexes before making opt file.
Fully handle blocked variable tables. Make opt file for blocked
variable tables even when they have no optimised colums.
modified:
storage/connect/tabdos.cpp
storage/connect/xindex.h
- Fix some errors making index
Return an error when the allocation is too small (should not
really occur now that GetMaxSize is sure)
Don't use XXROW index for DBF tables because of soft deleted lines.
modified:
storage/connect/xindex.cpp
- Typo
modified:
storage/connect/macutil.cpp
storage/connect/tabdos.h
storage/connect/tabsys.cpp
storage/connect/tabsys.h
were not taken in acount in check_if_supported_inplace_alter. mysql.test
failure is strange, the suppressed warning should not be made anyway.
modified:
storage/connect/ha_connect.cc
storage/connect/mysql-test/connect/r/mysql.result
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
tool chain
This is an addition to the original patch. On Windows
InterlockedExchange implies full memory barrier, whereas
only acquire/release barriers required.
buf0flu.cc line 549.
Analysis: If buf_page_get_state(bpage) == BUF_BLOCK_REMOVE_HASH then
buf_page_in_file(bpage) might not be true.
Fix: ut_a(buf_page_in_file(bpage) || buf_page_get_state(bpage) == BUF_BLOCK_REMOVE_HASH);
This bug only happens in case of paritioned tables used in LOCK TABLES and implicit_commit() was called
(as part of trying to execute a CREATE TABLE withing lock tables)
The problem was that Aria could not move the tables from one transaction to the new one, as thd->open_tables contained
a partitioned tables and not an Aria table.
Fix:
- Store a list of all open tables that are part of a share in share->open_tables
- In maria::implict_commit() use transaction->used_tables & share->open_tables to find out which tables
was part of the current transaction instead of using thd->open_tables, which may contain partitioned tables.
mysql-test/suite/maria/maria_partition.result:
Added test case
mysql-test/suite/maria/maria_partition.test:
Added test case
storage/maria/ha_maria.cc:
Use trn->used tables and share->open_tables to find out which tables was part of the current transaction instead of using thd->open_tables.
storage/maria/ma_close.c:
Remove closed table from share->open_list
storage/maria/ma_open.c:
Add table to share->open_list
storage/maria/ma_state.c:
Added comment
storage/maria/maria_def.h:
Added share->open_list, a list of all tables that is using this share.
InnoDB: Failing assertion: mutex_own(&buf_pool->LRU_list_mutex)
and
InnoDB: Assertion failure in thread 2868898624 in file buf0lru.c line 1077
InnoDB: Failing assertion: mutex_own(&buf_pool->LRU_list_mutex)
Analysis: Function buf_LRU_free_block might release LRU_list_mutex on
same cases to avoid mutex order problems, we need to take it back
before accessing list.
4229: MDEV-5670: Assertion failure in file buf0lru.c line 2355
Add more status information if repeatable.
4230: MDEV-5673: Crash while parallel dropping multiple tables under heavy load
Improve long semaphore wait output to include all semaphore waits
and try to find out if there is a sequence of waiters.
4233: Fix compiler errors on product build.
4237: Fix too agressive long semaphore wait output and add guard against introducing
compression failures on insert buffer.
4238: Fix test failure caused by simulated compression failure on
IBUF_DUMMY table.
This problem affects only debug builds on PPC64.
There are at least two race conditions around
rw_lock_debug_mutex_enter and rw_lock_debug_mutex_exit:
- rw_lock_debug_waiters was loaded/stored without setting
appropriate locks/memory barriers.
- there is a gap between calls to os_event_reset() and
os_event_wait() and in such case we're supposed to pass
return value of the former to the latter.
Fixed by replacing self-cooked spinlocks with system mutexes.
These days system mutexes offer much better performance. OTOH
performance is not that critical for debug builds.
This is to get same test results on Linux and Windows
modified:
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/mysql-test/connect/t/part_file.test
storage/connect/mysql-test/connect/t/part_table.test
storage/connect/tabdos.cpp
If mysql.innodb_table_stats or mysql.innodb_index_stats is not found or has
unexpected structure output that error only once and no other error for
every table trying to use them. If they do exists, then print fetch or
recalculation errors only once / table or index.
modified:
storage/connect/ha_connect.cc
storage/connect/tabdos.cpp
storage/connect/tabfmt.cpp
storage/connect/tabmysql.cpp
- add AVG_REC_LENGTH option to avoid result mismatch between
Windows and Linux
modified:
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/mysql-test/connect/t/part_file.test
line 8473
In case InnoDB index is not found, print the MySQL and InnoDB index
name we were trying to find and all MySQL and InnoDB index names there
is for this table.
ha_innodb.cc line 8473
If index is not found from InnoDB make sure we print what we
were trying to find and all mysql and InnoDB index names there
is for this table.
modified:
storage/connect/array.cpp
storage/connect/array.h
storage/connect/blkfil.cpp
storage/connect/blkfil.h
storage/connect/filter.cpp
storage/connect/filter.h
storage/connect/ha_connect.cc
storage/connect/tabdos.cpp
- FIX problems of DIR table on LINUX:
The order of the result is not the same than the one on Windows
An error occurs when no file match the pattern
Also the row numbers may also be different (not fixed yet)
modified:
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/mysql-test/connect/t/part_file.test
a sub-branch of this one and merged by pushing all the changes from it.
This version adds the following to CONNECT:
- MRR support (similar to the MyISAM one)
- Block, Remote and dynamic indexing
- Partitioning support (using the PARTITION engine)
Here is a list of the commited changes made in the sub-branch:
========================================================================
------------------------------------------------------------
revno: 4009
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Thu 2014-07-17 18:13:51 +0200
message:
This commit brings many changes, in particular two important ones:
1) Support of partitioning by connect. A table can be partitioned
by files, this is an enhanced MULTIPLE table. It can be also
partitioned by sub-tables like TBL and this enables table sharding.
2) Handling a CONNECT bug that causes in some cases extraneous rows
to remain in the table after an UPDATE or DELETE when the command
uses indexing (for not fixed file tables). Until a real fix is
done, CONNECT tries to ignore indexing and if it cannot do it
abort the command with an error message.
- Add tests on partitioning
added:
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/mysql-test/connect/t/part_file.test
storage/connect/mysql-test/connect/t/part_table.test
- Temporary fix
modified:
sql/sql_partition.cc
- Add partition support
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/reldef.cpp
storage/connect/reldef.h
storage/connect/tabdos.cpp
- Add functions ha_connect::IsUnique and ha_connect::CheckColumnList
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Prevent updating a partition table column that is part of
the partition function (outward tables only)
modified:
storage/connect/ha_connect.cc
- Support INSERT/UPDATE/DELETE for PROXY tables
modified:
storage/connect/tabutil.cpp
- Handle the bug on updating rows via indexing. Waiting for a real fix,
Don't use indexing when possible else raise an error and abort.
modified:
storage/connect/ha_connect.cc
- dbuserp->UseTemp set to TMP_AUTO
modified:
storage/connect/connect.cc
- Add members nox, abort and only
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Add arguments nox and abort to CntCloseTable
modified:
storage/connect/connect.cc
storage/connect/connect.h
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/ha_connect.cc
- Add arguments abort to CloseTableFile and RenameTempFile
modified:
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabvct.cpp
storage/connect/xtable.h
- Fix info->records when file does not exists
modified:
storage/connect/connect.cc
- Close XML table when opened for info
modified:
storage/connect/connect.cc
- Add function VCTFAM::GetFileLength
modified:
storage/connect/filamvct.cpp
storage/connect/filamvct.h
- Column option DISTRIB -> ENUM
modified:
storage/connect/ha_connect.cc
- Options connect, query_string and partname allways available
modified:
storage/connect/ha_connect.cc
- Add function MYSQLC::GetTableSize
modified:
storage/connect/myconn.cpp
storage/connect/myconn.h
- Add new special columns (PARTNAME, FNAME, FPATH, FTYPE and FDISK)
modified:
storage/connect/colblk.cpp
storage/connect/colblk.h
storage/connect/plgdbsem.h
storage/connect/table.cpp
- Add function ExtractFromPath
modified:
storage/connect/colblk.cpp
storage/connect/plgdbsem.h
storage/connect/plgdbutl.cpp
- Enhance Cardinality for some table types
modified:
storage/connect/tabdos.cpp
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
storage/connect/tabsys.cpp
storage/connect/tabsys.h
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
- Add test on special column
modified:
storage/connect/tabfmt.cpp
- Add new files (added for block indexing)
modified:
storage/connect/CMakeLists.txt
------------------------------------------------------------
revno: 4007 [merge]
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-05-31 12:31:26 +0200
message:
- Begin adding support of partition tables
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/reldef.cpp
- Add INSERT/UPDATE support to PROXY tables
modified:
storage/connect/tabutil.cpp
storage/connect/tabutil.h
- Take care of SPECIAL columns
modified:
storage/connect/filamdbf.cpp
storage/connect/reldef.h
storage/connect/tabfmt.cpp
-Typo and misc
modified:
storage/connect/odbconn.cpp
storage/connect/tabfix.cpp
storage/connect/xindex.cpp
------------------------------------------------------------
revno: 4006
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-05-10 12:21:08 +0200
message:
- FIX some MAP and XMAP errors (such as mapped indexes not closed)
Do not put version in XML files header
Remove HTON_NO_PARTITION for testing
Fix a wrong return (instead of DBUG_RETURN) in index_init
Plus a few typos
modified:
storage/connect/connect.cc
storage/connect/filter.cpp
storage/connect/ha_connect.cc
storage/connect/maputil.cpp
storage/connect/mysql-test/connect/r/alter_xml.result
storage/connect/mysql-test/connect/r/xml.result
storage/connect/table.cpp
storage/connect/tabxml.cpp
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
------------------------------------------------------------
revno: 4005
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Fri 2014-05-02 15:55:45 +0200
message:
- Adding fetched columns to Dynamic index key (unique only)
Fix two bugs concerning added KXYCOL's:
1 - Not set during reading
2 - Val_K not set in FastFind
modified:
storage/connect/connect.cc
storage/connect/filamtxt.h
storage/connect/tabdos.cpp
storage/connect/tabfix.cpp
storage/connect/table.cpp
storage/connect/valblk.h
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
------------------------------------------------------------
revno: 4003
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Wed 2014-04-30 10:48:29 +0200
message:
- Implementation of adding selected columns to dynamic indexes.
modified:
storage/connect/connect.cc
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabvct.cpp
storage/connect/tabvct.h
storage/connect/xindex.cpp
storage/connect/xindex.h
------------------------------------------------------------
revno: 4001
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sat 2014-04-26 00:17:26 +0200
message:
- Implement dynamic indexing
modified:
storage/connect/connect.cc
storage/connect/filter.cpp
storage/connect/filter.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/table.cpp
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
------------------------------------------------------------
revno: 3995
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Sun 2014-03-23 18:49:19 +0100
message:
- Work in progress
modified:
storage/connect/filter.h
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/mysql-test/connect/r/alter.result
storage/connect/mysql-test/connect/r/xml.result
------------------------------------------------------------
revno: 3991
committer: Olivier Bertrand <bertrandop@gmail.com>
branch nick: 10.0-connect
timestamp: Mon 2014-03-10 18:59:36 +0100
message:
- Adding files needed for block indexing
added:
storage/connect/array.cpp
storage/connect/array.h
storage/connect/blkfil.cpp
storage/connect/blkfil.h
storage/connect/filter.cpp
storage/connect/filter.h
========================================================================
This commit of the main branch adds:
- A change needed to have the engine function check_if_supported_inplace_alter
called for partition tables (was done manually in the sub-branch) by adding
the preparser define: PARTITION_SUPPORTS_INPLACE_ALTER
modified:
sql/CMakeLists.txt
- A fix concerning the FileExists function. It was needed to force the function
table_flags to return the same flags for all partitions. This is tested by
the partition engine and raises an error if flags are not equal.
The way file name, table name and connection string are retrieved has been
modified to cope with it.
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/reldef.cpp
- A few typos, such as the version string.
modified:
storage/connect/ha_connect.cc
- Updating some test result files because some warnings are no more raised.
modified:
storage/connect/mysql-test/connect/r/occur.result
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/pivot.result
Reason for the problem was that the hash of changed files in the key cache was too small (was 128). Fixed by making the hash size larger and changeable.
- Introduced key-cache-file-hash-size (default 512) for MyISAM and aria_pagecache_file_hash_size (default 512) for Aria.
- Added new status variable "Feature_delay_key_write" which counts number of tables opened that are using delay_key_write
mysql-test/r/features.result:
Added test of Feature_delay_key_write
mysql-test/r/key_cache.result:
Updated tests as the number of blocks has changed
mysql-test/r/mysqld--help.result:
Updated result
mysql-test/suite/maria/maria3.result:
Updated result
mysql-test/suite/sys_vars/r/key_cache_file_hash_size_basic.result:
Test new variable
mysql-test/suite/sys_vars/t/aria_pagecache_file_hash_size_basic.test:
Test new variable
mysql-test/suite/sys_vars/t/key_cache_file_hash_size_basic.test:
Test new variable
mysql-test/t/features.test:
Added test of Feature_delay_key_write
mysql-test/t/key_cache.test:
Updated tests as the number of blocks has changed
mysys/mf_keycache.c:
Made CHANGED_BLOCKS_HASH dynamic
sql/handler.cc:
Updated call to init_key_cache()
sql/mysqld.cc:
Added "Feature_delay_key_write"
Added support for key-cache-file-hash-size
sql/mysqld.h:
Added support for key-cache-file-hash-size
sql/sql_class.h:
Added feature_files_opened_with_delayed_keys
sql/sys_vars.cc:
Added key_cache_file_hash_size
storage/maria/ha_maria.cc:
Added pagecache_file_hash_size
Added counting of files with delay_key_write
storage/maria/ma_checkpoint.c:
Fixed compiler warning
storage/maria/ma_pagecache.c:
Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_pagecache.h:
Made PAGECACHE_CHANGED_BLOCKS_HASH into a variable
storage/maria/ma_rt_test.c:
Updated parameters for init_pagecache()
storage/maria/ma_test1.c:
Updated parameters for init_pagecache()
storage/maria/ma_test2.c:
Updated parameters for init_pagecache()
storage/maria/ma_test3.c:
Updated parameters for init_pagecache()
storage/maria/maria_chk.c:
Updated parameters for init_pagecache()
storage/maria/maria_ftdump.c:
Updated parameters for init_pagecache()
storage/maria/maria_pack.c:
Updated parameters for init_pagecache()
storage/maria/maria_read_log.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_consist.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_rwconsist2.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_pagecache_single.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
Updated parameters for init_pagecache()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
Updated parameters for init_pagecache()
storage/myisam/ha_myisam.cc:
Added counting of files with delay_key_write
storage/myisam/mi_check.c:
Updated call to init_key_cache()
storage/myisam/mi_test1.c:
Updated call to init_key_cache()
storage/myisam/mi_test2.c:
Updated call to init_key_cache()
storage/myisam/mi_test3.c:
Updated call to init_key_cache()
storage/myisam/mi_test_all.sh:
Fixed broken test
storage/myisam/myisam_ftdump.c:
Updated call to init_key_cache()
storage/myisam/myisamchk.c:
Updated call to init_key_cache()
storage/myisam/myisamlog.c:
Updated call to init_key_cache()
chain
InnoDB mutex_exit() function calls __sync_test_and_set() to release
the lock. According to manual this function is supposed to create
"acquire" memory barrier whereas in fact we need "release" memory
barrier at mutex_exit().
The problem isn't repeatable with gcc because it creates
"acquire-release" memory barrier for __sync_test_and_set().
ATC creates just "acquire" barrier.
Fixed by creating proper barrier at mutex_exit() by using
__sync_lock_release() instead of __sync_test_and_set().
1) Support of partitioning by connect. A table can be partitioned
by files, this is an enhanced MULTIPLE table. It can be also
partitioned by sub-tables like TBL and this enables table sharding.
2) Handling a CONNECT bug that causes in some cases extraneous rows
to remain in the table after an UPDATE or DELETE when the command
uses indexing (for not fixed file tables). Until a real fix is
done, CONNECT tries to ignore indexing and if it cannot do it
abort the command with an error message.
- Add tests on partitioning
added:
storage/connect/mysql-test/connect/r/part_file.result
storage/connect/mysql-test/connect/r/part_table.result
storage/connect/mysql-test/connect/t/part_file.test
storage/connect/mysql-test/connect/t/part_table.test
- Temporary fix
modified:
sql/sql_partition.cc
- Add partition support
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
storage/connect/reldef.cpp
storage/connect/reldef.h
storage/connect/tabdos.cpp
- Add functions ha_connect::IsUnique and ha_connect::CheckColumnList
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Prevent updating a partition table column that is part of
the partition function (outward tables only)
modified:
storage/connect/ha_connect.cc
- Support INSERT/UPDATE/DELETE for PROXY tables
modified:
storage/connect/tabutil.cpp
- Handle the bug on updating rows via indexing. Waiting for a real fix,
Don't use indexing when possible else raise an error and abort.
modified:
storage/connect/ha_connect.cc
- dbuserp->UseTemp set to TMP_AUTO
modified:
storage/connect/connect.cc
- Add members nox, abort and only
modified:
storage/connect/ha_connect.cc
storage/connect/ha_connect.h
- Add arguments nox and abort to CntCloseTable
modified:
storage/connect/connect.cc
storage/connect/connect.h
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/ha_connect.cc
- Add arguments abort to CloseTableFile and RenameTempFile
modified:
storage/connect/filamap.cpp
storage/connect/filamap.h
storage/connect/filamdbf.cpp
storage/connect/filamdbf.h
storage/connect/filamfix.cpp
storage/connect/filamfix.h
storage/connect/filamtxt.cpp
storage/connect/filamtxt.h
storage/connect/filamvct.cpp
storage/connect/filamvct.h
storage/connect/filamzip.cpp
storage/connect/filamzip.h
storage/connect/tabdos.cpp
storage/connect/tabdos.h
storage/connect/tabvct.cpp
storage/connect/xtable.h
- Fix info->records when file does not exists
modified:
storage/connect/connect.cc
- Close XML table when opened for info
modified:
storage/connect/connect.cc
- Add function VCTFAM::GetFileLength
modified:
storage/connect/filamvct.cpp
storage/connect/filamvct.h
- Column option DISTRIB -> ENUM
modified:
storage/connect/ha_connect.cc
- Options connect, query_string and partname allways available
modified:
storage/connect/ha_connect.cc
- Add function MYSQLC::GetTableSize
modified:
storage/connect/myconn.cpp
storage/connect/myconn.h
- Add new special columns (PARTNAME, FNAME, FPATH, FTYPE and FDISK)
modified:
storage/connect/colblk.cpp
storage/connect/colblk.h
storage/connect/plgdbsem.h
storage/connect/table.cpp
- Add function ExtractFromPath
modified:
storage/connect/colblk.cpp
storage/connect/plgdbsem.h
storage/connect/plgdbutl.cpp
- Enhance Cardinality for some table types
modified:
storage/connect/tabdos.cpp
storage/connect/tabmysql.cpp
storage/connect/tabmysql.h
storage/connect/tabodbc.cpp
storage/connect/tabodbc.h
storage/connect/tabsys.cpp
storage/connect/tabsys.h
storage/connect/xindex.cpp
storage/connect/xindex.h
storage/connect/xtable.h
- Add test on special column
modified:
storage/connect/tabfmt.cpp
- Add new files (added for block indexing)
modified:
storage/connect/CMakeLists.txt
Merge the patches into MariaDB 10.0 main.
With this patch, parallel replication will now automatically retry a
transaction that fails due to deadlock or other temporary error, same as
single-threaded replication.
We catch deadlocks with InnoDB transactions due to enforced commit order. If
T1 must commit before T2 in parallel replication and T1 ends up waiting for T2
inside InnoDB, we kill T2 and retry it later to resolve the deadlock
automatically.
After-review changes.
For this patch in 10.0, we do not introduce a new public storage engine API,
we just fix the InnoDB/XtraDB issues. In 10.1, we will make a better public
API that can be used for all storage engines (MDEV-6429).
Eliminate the background thread that did deadlock kills asynchroneously.
Instead, we ensure that the InnoDB/XtraDB code can handle doing the kill from
inside the deadlock detection code (when thd_report_wait_for() needs to kill a
later thread to resolve a deadlock).
(We preserve the part of the original patch that introduces dedicated mutex
and condition for the slave init thread, to remove the abuse of
LOCK_thread_count for start/stop synchronisation of the slave init thread).
IN RECOVERY
During redo log processing, the data dictionary is not available. We should
check it in dict_find_table_by_space() to prevent SEGV error.
rb#5678, approved by Jimmy.
Problem:
When a unique secondary index is scanned for duplicate checking, gap locks
were not taken if the transaction had isolation level <= READ COMMITTED.
This change was done while fixing Bug #16133801 UNEXPLAINABLE INNODB UNIQUE
INDEX LOCKS ON DELETE + INSERT WITH SAME VALUES (rb#2035). Because of this
the duplicate check logic failed, and resulted in duplicate values in unique
secondary index.
Solution:
When a unique secondary index is scanned for duplicate checking, gap locks
must be taken irrespective of the transaction isolation level. This is
achieved by reverting rb#2035.
rb#5910 approved by Jimmy
Analysis: For some reason table stats for a table pointed from a index
is not initialized. Added additional warning output on this situation
and table stats initialization. This is better than asserting.
than with InnoDB plugin
Fix: os0file.h in XtraDB had OS_AIO_N_PENDING_IOS_PER_THREAD 256
when on InnoDB it is OS_AIO_N_PENDING_IOS_PER_THREAD 32. Changed
XtraDB also to use 32.
WITH CERTAIN MAX_HEAP_TABLE_SIZE VALUES
Description:
When the system variable 'max_heap_table_size'
is set to 20GB, the server crashes on creation of a
temporary tables or tables using MEMORY storage engine.
Analysis:
The variable 'max_record' determines the amount heap
allocated for the records of the table. This value
is determined using the 'max_heap_table_size' variable.
'records_in_block' in turn uses the max_records to
determine the number of records per block.
When the 'max_heap_table_size' is set to 20GB, then
the 'records_in_block' is calculated to a value of
2^28.
The size of the block determined by multiplying the
'records_in_block' and 'recbuffer' results in overflow
and hence the value becomes zero. As a result, zero bytes
of the heap is allocated for the table. This will
result in a server crash when the table is accessed.
Fix:
The variables 'records_in_block' and 'recbuffer' are
typecasted to 'unsigned long' while calculating the
size of the block.
than with InnoDB plugin
Fix: os0file.h in XtraDB had OS_AIO_N_PENDING_IOS_PER_THREAD 256
when on InnoDB it is OS_AIO_N_PENDING_IOS_PER_THREAD 32. Changed
XtraDB also to use 32.
Analysis: Based on crashed the buffer pool instance identifier is
not correct on block to be freed. Add LRU list mutex holding
on functions calling free and add additional safety checks.
