d145d1b6ee
Merge into the MariaDB tree the pull request from Rich Prohaska for
PerconaFT. These changes are needed to get parallel replication to
work with TokuDB. Once the pull request is accepted by Percona and the new upstream version enters MariaDB, this commit can be superseded.
Original commit message from Rich Prohaska:
1. Fix the release before wait race
The release before wait race occurs when a lock is released by transaction A after transaction B tried to acquire it but before transaction B has a chance to register it's pending lock request. There are several ways to fix this problem, but we want to optimize for the common situation of minimal lock conflicts, which is what the lock acquisition algorithm currently does. Our solution to the release before wait race is for transaction B to retry its lock request after its lock request has been added to the pending lock set.
2. Fix the retry race
The retry race occurs in the current lock retry algorithm which assumes that if some transaction is running lock retry, then my transaction does not also need to run it. There is a chance that some pending lock requests will be skipped, but these lock requests will eventually time out. For applications with small numbers of concurrent transactions, timeouts will frequently occur, and the application throughput will be very small.
The solution to the retry race is to use a group retry algorithm. All threads run through the retry logic. Sequence numbers are used to group retries into batches such that one transaction can run the retry logic on behalf of several transactions. This amortizes the retry cost. The sequence numbers also ensure that when a transaction releases its locks, all of the pending lock requests that it is blocking are retried.
3. Implement a mechanism to find and kill a pending lock request
Tags lock requests with a client id, use the client id as a key into the pending lock requests sets to find a lock request, complete the lock request with a lock timeout error.
Copyright (c) 2016, Rich Prohaska
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||
|---|---|---|
| .. | ||
| buildbot | ||
| buildheader | ||
| cmake | ||
| cmake_modules | ||
| ft | ||
| ftcxx | ||
| locktree | ||
| portability | ||
| scripts | ||
| src | ||
| third_party | ||
| tools | ||
| util | ||
| bash.suppressions | ||
| CMakeLists.txt | ||
| COPYING.AGPLv3 | ||
| COPYING.GPLv2 | ||
| CTestConfig.cmake | ||
| CTestCustom.cmake.in | ||
| DartConfig.cmake | ||
| PATENTS | ||
| README.md | ||
PerconaFT
PerconaFT is a high-performance, transactional key-value store, used in the TokuDB storage engine for Percona Server and MySQL, and in TokuMX, the high-performance MongoDB distribution.
PerconaFT is provided as a shared library with an interface similar to Berkeley DB.
To build the full MySQL product, see the instructions for Percona/tokudb-engine. To build TokuMX, see the instructions for Percona/percona-server-mongodb. This document covers PerconaFT only.
Building
PerconaFT is built using CMake >= 2.8.9. Out-of-source builds are
recommended. You need a C++11 compiler, though only GCC >= 4.7 and
Apple's Clang are tested. You also need zlib development packages
(yum install zlib-devel or apt-get install zlib1g-dev).
You will also need the source code for jemalloc, checked out in
third_party/.
git clone git://github.com/Percona/PerconaFT.git percona-ft
cd percona-ft
git clone git://github.com/Percona/jemalloc.git third_party/jemalloc
mkdir build
cd build
CC=gcc47 CXX=g++47 cmake \
-D CMAKE_BUILD_TYPE=Debug \
-D BUILD_TESTING=OFF \
-D USE_VALGRIND=OFF \
-D CMAKE_INSTALL_PREFIX=../prefix/ \
..
cmake --build . --target install
This will build libtokudb.so and libtokuportability.so and install it,
some header files, and some examples to percona-ft/prefix/. It will also
build jemalloc and install it alongside these libraries, you should link
to that if you are planning to run benchmarks or in production.
Platforms
PerconaFT is supported on 64-bit Centos, should work on other 64-bit linux distributions, and may work on OSX 10.8 and FreeBSD. PerconaFT is not supported on 32-bit systems.
Transparent hugepages is a feature in newer linux
kernel versions that causes problems for the memory usage tracking
calculations in PerconaFT and can lead to memory overcommit. If you have
this feature enabled, PerconaFT will not start, and you should turn it off.
If you want to run with transparent hugepages on, you can set an
environment variable TOKU_HUGE_PAGES_OK=1, but only do this for testing,
and only with a small cache size.
Testing
PerconaFT uses CTest for testing. The CDash testing dashboard is not currently public, but you can run the tests without submitting them.
There are some large data files not stored in the git repository, that will be made available soon. For now, the tests that use these files will not run.
In the build directory from above:
cmake -D BUILD_TESTING=ON ..
ctest -D ExperimentalStart \
-D ExperimentalConfigure \
-D ExperimentalBuild \
-D ExperimentalTest
Contributing
Please report bugs in PerconaFT to the issue tracker.
We have two publicly accessible mailing lists for TokuDB:
- tokudb-user@googlegroups.com is for general and support related questions about the use of TokuDB.
- tokudb-dev@googlegroups.com is for discussion of the development of TokuDB.
and two for TokuMX:
- tokumx-user@googlegroups.com is for general and support related questions about the use of TokuMX.
- tokumx-dev@googlegroups.com is for discussion of the development of TokuMX.
All source code and test contributions must be provided under a BSD 2-Clause license. For any small change set, the license text may be contained within the commit comment and the pull request. For larger contributions, the license must be presented in a COPYING.<feature_name> file in the root of the PerconaFT project. Please see the BSD 2-Clause license template for the content of the license text.
License
PerconaFT is available under the GPL version 2, and AGPL version 3. See COPYING.AGPLv3, COPYING.GPLv2, and PATENTS.