mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-08-03 09:01:34 +02:00
Code Issues Projects Releases Packages Wiki Activity
mariadb/storage/innobase/include/rem0cmp.h
Marko Mäkelä e245c6ab11 MDEV-35049: Use CRC-32C and avoid allocating heap 
For the adaptive hash index, dtuple_fold() and rec_fold() were employing
a slow rolling hash algorithm, computing hash values ("fold") for one
field and one byte at a time, while depending on calls to
rec_get_offsets().

We already have optimized implementations of CRC-32C and have been
successfully using that function in some other InnoDB tables, but not
yet in the adaptive hash index.

Any linear function such as any CRC will fail the avalanche test that
any cryptographically secure hash function is expected to pass:
any single-bit change in the input key should affect on average half
the bits in the output.

But we always were happy with less than cryptographically secure:
in fact, ut_fold_ulint_pair() or ut_fold_binary() are just about as
linear as any CRC, using a combination of multiplication and addition,
partly carry-less. It is worth noting that exclusive-or corresponds to
carry-less subtraction or addition in a binary Galois field, or GF(2).

We only need some way of reducing key prefixes into hash values.
The CRC-32C should be better than a Rabin–Karp rolling hash algorithm.
Compared to the old hash algorithm, it has the drawback that there will
be only 32 bits of entropy before we choose the hash table cell by a
modulus operation. The size of each adaptive hash index array is
(innodb_buffer_pool_size / 512) / innodb_adaptive_hash_index_parts.
With the maximum number of partitions (512), we would not exceed 1<<32
elements per array until the buffer pool size exceeds 1<<50 bytes (1 PiB).
We would hit other limits before that: the virtual address space on many
contemporary 64-bit processor implementations is only 48 bits (256 TiB).
So, we can simply go for the SIMD accelerated CRC-32C.

rec_fold(): Take a combined parameter n_bytes_fields. Determine the
length of each field on the fly, and compute CRC-32C over a single
contiguous range of bytes, from the start of the record payload area
to the end of the last full or partial field. For secondary index records
in ROW_FORMAT=REDUNDANT, also the data area that is reserved for NULL
values (to facilitate in-place updates between NULL and NOT NULL values)
will be included in the count. Luckily, InnoDB always zero-initialized
such unused area; refer to data_write_sql_null() in
rec_convert_dtuple_to_rec_old(). For other than ROW_FORMAT=REDUNDANT,
no space is allocated for NULL values, and therefore the CRC-32C will
only cover the actual payload of the key prefix.

dtuple_fold(): For ROW_FORMAT=REDUNDANT, include the dummy NULL values
in the CRC-32C, so that the values will be comparable with rec_fold().

innodb_ahi-t: A unit test for rec_fold() and dtuple_fold().

btr_search_build_page_hash_index(), btr_search_drop_page_hash_index():
Use a fixed-size stack buffer for computing the fold values, to avoid
dynamic memory allocation.

btr_search_drop_page_hash_index(): Do not release part.latch if we
need to invoke multiple batches of rec_fold().

dtuple_t: Allocate fewer bits for the fields. The maximum number of
data fields is about 1023, so uint16_t will be fine for them. The
info_bits is stored in less than 1 byte.

ut_pair_min(), ut_pair_cmp(): Remove. We can actually combine and compare
int(n_fields << 16 | n_bytes).

PAGE_CUR_LE_OR_EXTENDS, PAGE_CUR_DBG: Remove. These were never defined,
because they would only work with latin1_swedish_ci if at all.

btr_cur_t::check_mismatch(): Replaces !btr_search_check_guess().

cmp_dtuple_rec_bytes(): Replaces cmp_dtuple_rec_with_match_bytes().
Determine the offsets of fields on the fly.

page_cur_try_search_shortcut_bytes(): This caller of
cmp_dtuple_rec_bytes() will not be invoked on the change buffer tree.

cmp_dtuple_rec_leaf(): Replaces cmp_dtuple_rec_with_match()
for comparing leaf-page records.

buf_block_t::ahi_left_bytes_fields: Consolidated Atomic_relaxed<uint32_t>
of curr_left_side << 31 | curr_n_bytes << 16 | curr_n_fields.
The other set of parameters (n_fields, n_bytes, left_side) was removed
as redundant.

btr_search_update_hash_node_on_insert(): Merged to
btr_search_update_hash_on_insert().

btr_search_build_page_hash_index(): Take combined left_bytes_fields
instead of n_fields, n_bytes, left_side.

btr_search_update_block_hash_info(), btr_search_update_hash_ref():
Merged to btr_search_info_update_hash().

btr_cur_t::n_bytes_fields: Replaces n_bytes << 16 | n_fields.

We also remove many redundant checks of btr_search.enabled.
If we are holding any btr_sea::partition::latch, then a nonnull pointer
in buf_block_t::index must imply that the adaptive hash index is enabled.

Reviewed by: Vladislav Lesin
2025-01-10 14:24:01 +02:00

237 lines
8.1 KiB
C
Raw Permalink Blame History

/*****************************************************************************
Copyright (c) 1994, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2017, 2020, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA
*****************************************************************************/
/*******************************************************************//**
@file include/rem0cmp.h
Comparison services for records
Created 7/1/1994 Heikki Tuuri
************************************************************************/
#ifndef rem0cmp_h
#define rem0cmp_h
#include "data0data.h"
#include "data0type.h"
#include "rem0types.h"
#include "page0types.h"
/*************************************************************//**
Returns TRUE if two columns are equal for comparison purposes.
@return TRUE if the columns are considered equal in comparisons */
ibool
cmp_cols_are_equal(
/*===============*/
const dict_col_t* col1, /*!< in: column 1 */
const dict_col_t* col2, /*!< in: column 2 */
ibool check_charsets);
/*!< in: whether to check charsets */
/** Compare two data fields.
@param mtype main type
@param prtype precise type
@param data1 data field
@param len1 length of data1 in bytes, or UNIV_SQL_NULL
@param data2 data field
@param len2 length of data2 in bytes, or UNIV_SQL_NULL
@return the comparison result of data1 and data2
@retval 0 if data1 is equal to data2
@retval negative if data1 is less than data2
@retval positive if data1 is greater than data2 */
int cmp_data_data(ulint mtype, ulint prtype, const byte *data1, ulint len1,
const byte *data2, ulint len2) noexcept
MY_ATTRIBUTE((warn_unused_result));
/** Compare two data fields.
@param[in] dfield1 data field; must have type field set
@param[in] dfield2 data field
@return the comparison result of dfield1 and dfield2
@retval 0 if dfield1 is equal to dfield2
@retval negative if dfield1 is less than dfield2
@retval positive if dfield1 is greater than dfield2 */
UNIV_INLINE
int
cmp_dfield_dfield(
/*==============*/
const dfield_t* dfield1,/*!< in: data field; must have type field set */
const dfield_t* dfield2);/*!< in: data field */
#ifdef UNIV_DEBUG
/** Compare a GIS data tuple to a physical record.
@param[in] dtuple data tuple
@param[in] rec R-tree record
@param[in] mode compare mode
@retval negative if dtuple is less than rec */
int cmp_dtuple_rec_with_gis(const dtuple_t *dtuple, const rec_t *rec,
page_cur_mode_t mode)
MY_ATTRIBUTE((nonnull));
#endif
/** Compare two minimum bounding rectangles.
@return 1, 0, -1, if a is greater, equal, less than b, respectively */
inline int cmp_geometry_field(const void *a, const void *b)
{
const byte *mbr1= static_cast<const byte*>(a);
const byte *mbr2= static_cast<const byte*>(b);
static_assert(SPDIMS == 2, "compatibility");
static_assert(DATA_MBR_LEN == SPDIMS * 2 * sizeof(double), "compatibility");
/* Try to compare mbr left lower corner (xmin, ymin) */
double x1= mach_double_read(mbr1);
double x2= mach_double_read(mbr2);
if (x1 > x2)
return 1;
if (x2 > x1)
return -1;
double y1= mach_double_read(mbr1 + sizeof(double) * SPDIMS);
double y2= mach_double_read(mbr2 + sizeof(double) * SPDIMS);
if (y1 > y2)
return 1;
if (y2 > y1)
return -1;
/* left lower corner (xmin, ymin) overlaps, now right upper corner */
x1= mach_double_read(mbr1 + sizeof(double));
x2= mach_double_read(mbr2 + sizeof(double));
if (x1 > x2)
return 1;
if (x2 > x1)
return -1;
y1= mach_double_read(mbr1 + sizeof(double) * 2 + sizeof(double));
y2= mach_double_read(mbr2 + sizeof(double) * 2 + sizeof(double));
if (y1 > y2)
return 1;
if (y2 > y1)
return -1;
return 0;
}
/** Compare a data tuple to a physical record.
@param[in] dtuple data tuple
@param[in] rec B-tree record
@param[in] offsets rec_get_offsets(rec)
@param[in] n_cmp number of fields to compare
@param[in,out] matched_fields number of completely matched fields
@return the comparison result of dtuple and rec
@retval 0 if dtuple is equal to rec
@retval negative if dtuple is less than rec
@retval positive if dtuple is greater than rec */
int
cmp_dtuple_rec_with_match_low(
const dtuple_t* dtuple,
const rec_t* rec,
const rec_offs* offsets,
ulint n_cmp,
uint16_t* matched_fields)
MY_ATTRIBUTE((nonnull));
#define cmp_dtuple_rec_with_match(tuple,rec,offsets,fields) \
cmp_dtuple_rec_with_match_low( \
tuple,rec,offsets,dtuple_get_n_fields_cmp(tuple),fields)
/** Compare a data tuple to a physical record.
@see cmp_dtuple_rec_with_match
@param[in] dtuple data tuple
@param[in] rec B-tree record
@param[in] offsets rec_get_offsets(rec)
@return the comparison result of dtuple and rec
@retval 0 if dtuple is equal to rec
@retval negative if dtuple is less than rec
@retval positive if dtuple is greater than rec */
int
cmp_dtuple_rec(
const dtuple_t* dtuple,
const rec_t* rec,
const rec_offs* offsets);
/**************************************************************//**
Checks if a dtuple is a prefix of a record. The last field in dtuple
is allowed to be a prefix of the corresponding field in the record.
@return TRUE if prefix */
ibool
cmp_dtuple_is_prefix_of_rec(
/*========================*/
const dtuple_t* dtuple, /*!< in: data tuple */
const rec_t* rec, /*!< in: physical record */
const rec_offs* offsets);/*!< in: array returned by rec_get_offsets() */
/** Compare two physical records that contain the same number of columns,
none of which are stored externally.
@retval positive if rec1 (including non-ordering columns) is greater than rec2
@retval negative if rec1 (including non-ordering columns) is less than rec2
@retval 0 if rec1 is a duplicate of rec2 */
int
cmp_rec_rec_simple(
/*===============*/
const rec_t* rec1, /*!< in: physical record */
const rec_t* rec2, /*!< in: physical record */
const rec_offs* offsets1,/*!< in: rec_get_offsets(rec1, ...) */
const rec_offs* offsets2,/*!< in: rec_get_offsets(rec2, ...) */
const dict_index_t* index, /*!< in: data dictionary index */
struct TABLE* table) /*!< in: MySQL table, for reporting
duplicate key value if applicable,
or NULL */
MY_ATTRIBUTE((nonnull(1,2,3,4), warn_unused_result));
/** Compare two B-tree or R-tree records.
Only the common first fields are compared, and externally stored field
are treated as equal.
@param[in] rec1 record (possibly not on an index page)
@param[in] rec2 B-tree or R-tree record in an index page
@param[in] offsets1 rec_get_offsets(rec1, index)
@param[in] offsets2 rec_get_offsets(rec2, index)
@param[in] nulls_unequal true if this is for index cardinality
statistics estimation with
innodb_stats_method=nulls_unequal
or innodb_stats_method=nulls_ignored
@param[out] matched_fields number of completely matched fields
within the first field not completely matched
@retval 0 if rec1 is equal to rec2
@retval negative if rec1 is less than rec2
@retval positive if rec1 is greater than rec2 */
int
cmp_rec_rec(
const rec_t* rec1,
const rec_t* rec2,
const rec_offs* offsets1,
const rec_offs* offsets2,
const dict_index_t* index,
bool nulls_unequal = false,
ulint* matched_fields = NULL)
MY_ATTRIBUTE((nonnull(1,2,3,4,5)));
/** Compare two data fields.
@param[in] dfield1 data field
@param[in] dfield2 data field
@return the comparison result of dfield1 and dfield2
@retval 0 if dfield1 is equal to dfield2, or a prefix of dfield1
@retval negative if dfield1 is less than dfield2
@retval positive if dfield1 is greater than dfield2 */
UNIV_INLINE
int
cmp_dfield_dfield_like_prefix(
const dfield_t* dfield1,
const dfield_t* dfield2);
#include "rem0cmp.inl"
#endif
Reference in a new issue View git blame Copy permalink