4dcb1b575b
For the adaptive hash index, dtuple_fold() and rec_fold() were employing
a slow rolling hash algorithm, computing hash values ("fold") for one
field and one byte at a time, while depending on calls to
rec_get_offsets().
We already have optimized implementations of CRC-32C and have been
successfully using that function in some other InnoDB tables, but not
yet in the adaptive hash index.
Any linear function such as any CRC will fail the avalanche test that
any cryptographically secure hash function is expected to pass:
any single-bit change in the input key should affect on average half
the bits in the output.
But we always were happy with less than cryptographically secure:
in fact, ut_fold_ulint_pair() or ut_fold_binary() are just about as
linear as any CRC, using a combination of multiplication and addition,
partly carry-less. It is worth noting that exclusive-or corresponds to
carry-less subtraction or addition in a binary Galois field, or GF(2).
We only need some way of reducing key prefixes into hash values.
The CRC-32C should be better than a Rabin–Karp rolling hash algorithm.
Compared to the old hash algorithm, it has the drawback that there will
be only 32 bits of entropy before we choose the hash table cell by a
modulus operation. The size of each adaptive hash index array is
(innodb_buffer_pool_size / 512) / innodb_adaptive_hash_index_parts.
With the maximum number of partitions (512), we would not exceed 1<<32
elements per array until the buffer pool size exceeds 1<<50 bytes (1 PiB).
We would hit other limits before that: the virtual address space on many
contemporary 64-bit processor implementations is only 48 bits (256 TiB).
So, we can simply go for the SIMD accelerated CRC-32C.
rec_fold(): Take a combined parameter n_bytes_fields. Determine the
length of each field on the fly, and compute CRC-32C over a single
contiguous range of bytes, from the start of the record payload area
to the end of the last full or partial field. For secondary index records
in ROW_FORMAT=REDUNDANT, also the data area that is reserved for NULL
values (to facilitate in-place updates between NULL and NOT NULL values)
will be included in the count. Luckily, InnoDB always zero-initialized
such unused area; refer to data_write_sql_null() in
rec_convert_dtuple_to_rec_old(). For other than ROW_FORMAT=REDUNDANT,
no space is allocated for NULL values, and therefore the CRC-32C will
only cover the actual payload of the key prefix.
dtuple_fold(): For ROW_FORMAT=REDUNDANT, include the dummy NULL values
in the CRC-32C, so that the values will be comparable with rec_fold().
innodb_ahi-t: A unit test for rec_fold() and dtuple_fold().
btr_search_build_page_hash_index(), btr_search_drop_page_hash_index():
Use a fixed-size stack buffer for computing the fold values, to avoid
dynamic memory allocation.
btr_search_drop_page_hash_index(): Do not release part.latch if we
need to invoke multiple batches of rec_fold().
dtuple_t: Allocate fewer bits for the fields. The maximum number of
data fields is about 1023, so uint16_t will be fine for them. The
info_bits is stored in less than 1 byte.
ut_pair_min(), ut_pair_cmp(): Remove. We can actually combine and compare
int(n_fields << 16 | n_bytes).
PAGE_CUR_LE_OR_EXTENDS, PAGE_CUR_DBG: Remove. These were never defined,
because they would only work with latin1_swedish_ci if at all.
btr_cur_t::check_mismatch(): Replaces !btr_search_check_guess().
cmp_dtuple_rec_bytes(): Replaces cmp_dtuple_rec_with_match_bytes().
Determine the offsets of fields on the fly.
page_cur_try_search_shortcut_bytes(): This caller of
cmp_dtuple_rec_bytes() will not be invoked on the change buffer tree.
cmp_dtuple_rec_leaf(): Replaces cmp_dtuple_rec_with_match()
for comparing leaf-page records.
buf_block_t::ahi_left_bytes_fields: Consolidated Atomic_relaxed<uint32_t>
of curr_left_side << 31 | curr_n_bytes << 16 | curr_n_fields.
The other set of parameters (n_fields, n_bytes, left_side) was removed
as redundant.
btr_search_update_hash_node_on_insert(): Merged to
btr_search_update_hash_on_insert().
btr_search_build_page_hash_index(): Take combined left_bytes_fields
instead of n_fields, n_bytes, left_side.
btr_search_update_block_hash_info(), btr_search_update_hash_ref():
Merged to btr_search_info_update_hash().
btr_cur_t::n_bytes_fields: Replaces n_bytes << 16 | n_fields.
We also remove many redundant checks of btr_search.enabled.
If we are holding any btr_sea::partition::latch, then a nonnull pointer
in buf_block_t::index must imply that the adaptive hash index is enabled.
Reviewed by: Vladislav Lesin
|
||
|---|---|---|
| .github | ||
| BUILD | ||
| client | ||
| cmake | ||
| dbug | ||
| debian | ||
| Docs | ||
| extra | ||
| include | ||
| libmariadb@52d0a38ed1 | ||
| libmysqld | ||
| libservices | ||
| man | ||
| mysql-test | ||
| mysys | ||
| mysys_ssl | ||
| plugin | ||
| randgen/conf | ||
| scripts | ||
| sql | ||
| sql-bench | ||
| sql-common | ||
| storage | ||
| strings | ||
| support-files | ||
| tests | ||
| tpool | ||
| unittest | ||
| vio | ||
| win | ||
| wsrep-lib@70cd967f5e | ||
| zlib | ||
| .clang-format | ||
| .gitattributes | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .gitmodules | ||
| appveyor.yml | ||
| BUILD-CMAKE | ||
| CMakeLists.txt | ||
| CODING_STANDARDS.md | ||
| config.h.cmake | ||
| configure.cmake | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| CREDITS | ||
| INSTALL-SOURCE | ||
| INSTALL-WIN-SOURCE | ||
| KNOWN_BUGS.txt | ||
| README.md | ||
| SECURITY.md | ||
| THIRDPARTY | ||
| VERSION | ||
Code status:
ci.appveyor.com
MariaDB: The innovative open source database
MariaDB was designed as a drop-in replacement of MySQL(R) with more features, new storage engines, fewer bugs, and better performance.
MariaDB is brought to you by the MariaDB Foundation and the MariaDB Corporation. Please read the CREDITS file for details about the MariaDB Foundation, and who is developing MariaDB.
MariaDB is developed by many of the original developers of MySQL who now work for the MariaDB Corporation, the MariaDB Foundation and by many people in the community.
MySQL, which is the base of MariaDB, is a product and trademark of Oracle Corporation, Inc. For a list of developers and other contributors, see the Credits appendix. You can also run 'SHOW authors' to get a list of active contributors.
A description of the MariaDB project and a manual can be found at:
https://mariadb.com/kb/en/mariadb-vs-mysql-features/
https://mariadb.com/kb/en/mariadb-versus-mysql-compatibility/
https://mariadb.com/kb/en/new-and-old-releases/
Getting the code, building it and testing it
Refer to the following guide: https://mariadb.org/get-involved/getting-started-for-developers/get-code-build-test/ which outlines how to build the source code correctly and run the MariaDB testing framework, as well as which branch to target for your contributions.
Help
More help is available from the Maria Discuss mailing list https://lists.mariadb.org/postorius/lists/discuss.lists.mariadb.org/ and MariaDB's Zulip instance, https://mariadb.zulipchat.com/
Licensing
MariaDB is specifically available only under version 2 of the GNU General Public License (GPLv2). (I.e. Without the "any later version" clause.) This is inherited from MySQL. Please see the README file in the MySQL distribution for more information.
License information can be found in the COPYING file. Third party license information can be found in the THIRDPARTY file.
Bug Reports
Bug and/or error reports regarding MariaDB should be submitted at: https://jira.mariadb.org
For reporting security vulnerabilities, see our security-policy.
The code for MariaDB, including all revision history, can be found at: https://github.com/MariaDB/server