mirror of
https://github.com/MariaDB/server.git
synced 2025-01-16 03:52:35 +01:00
7a4fbb55b0
Historically, InnoDB supported a buggy page checksum algorithm that did not
compute a checksum over the full page. Later, well before MySQL 4.1
introduced .ibd files and the innodb_file_per_table option, the algorithm
was corrected and the first 4 bytes of each page were redefined to be
a checksum.
The original checksum was so slow that an option to disable page checksum
was introduced for benchmarketing purposes.
The Intel Nehalem microarchitecture introduced the SSE4.2 instruction set
extension, which includes instructions for faster computation of CRC-32C.
In MySQL 5.6 (and MariaDB 10.0), innodb_checksum_algorithm=crc32 was
implemented to make of that. As that option was changed to be the default
in MySQL 5.7, a bug was found on big-endian platforms and some work-around
code was added to weaken that checksum further. MariaDB disables that
work-around by default since MDEV-17958.
Later, SIMD-accelerated CRC-32C has been implemented in MariaDB for POWER
and ARM and also for IA-32/AMD64, making use of carry-less multiplication
where available.
Long story short, innodb_checksum_algorithm=crc32 is faster and more secure
than the pre-MySQL 5.6 checksum, called innodb_checksum_algorithm=innodb.
It should have removed any need to use innodb_checksum_algorithm=none.
The setting innodb_checksum_algorithm=crc32 is the default in
MySQL 5.7 and MariaDB Server 10.2, 10.3, 10.4. In MariaDB 10.5,
MDEV-19534 made innodb_checksum_algorithm=full_crc32 the default.
It is even faster and more secure.
The default settings in MariaDB do allow old data files to be read,
no matter if a worse checksum algorithm had been used.
(Unfortunately, before innodb_checksum_algorithm=full_crc32,
the data files did not identify which checksum algorithm is being used.)
The non-default settings innodb_checksum_algorithm=strict_crc32 or
innodb_checksum_algorithm=strict_full_crc32 would only allow CRC-32C
checksums. The incompatibility with old data files is why they are
not the default.
The newest server not to support innodb_checksum_algorithm=crc32
were MySQL 5.5 and MariaDB 5.5. Both have reached their end of life.
A valid reason for using innodb_checksum_algorithm=innodb could have
been the ability to downgrade. If it is really needed, data files
can be converted with an older version of the innochecksum utility.
Because there is no good reason to allow data files to be written
with insecure checksums, we will reject those option values:
innodb_checksum_algorithm=none
innodb_checksum_algorithm=innodb
innodb_checksum_algorithm=strict_none
innodb_checksum_algorithm=strict_innodb
Furthermore, the following innochecksum options will be removed,
because only strict crc32 will be supported:
innochecksum --strict-check=crc32
innochecksum -C crc32
innochecksum --write=crc32
innochecksum -w crc32
If a user wishes to convert a data file to use a different checksum
(so that it might be used with the no-longer-supported
MySQL 5.5 or MariaDB 5.5, which do not support IMPORT TABLESPACE
nor system tablespace format changes that were made in MariaDB 10.3),
then the innochecksum tool from MariaDB 10.2, 10.3, 10.4, 10.5 or
MySQL 5.7 can be used.
Reviewed by: Thirunarayanan Balathandayuthapani
99 lines
3.7 KiB
C++
99 lines
3.7 KiB
C++
/*****************************************************************************
|
|
|
|
Copyright (c) 1995, 2016, Oracle and/or its affiliates. All Rights Reserved.
|
|
Copyright (c) 2017, 2021, MariaDB Corporation.
|
|
|
|
This program is free software; you can redistribute it and/or modify it under
|
|
the terms of the GNU General Public License as published by the Free Software
|
|
Foundation; version 2 of the License.
|
|
|
|
This program is distributed in the hope that it will be useful, but WITHOUT
|
|
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
|
|
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License along with
|
|
this program; if not, write to the Free Software Foundation, Inc.,
|
|
51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA
|
|
|
|
*****************************************************************************/
|
|
|
|
/**************************************************//**
|
|
@file buf/buf0checksum.cc
|
|
Buffer pool checksum functions, also linked from /extra/innochecksum.cc
|
|
|
|
Created Aug 11, 2011 Vasil Dimov
|
|
*******************************************************/
|
|
|
|
#include "buf0checksum.h"
|
|
#include "fil0fil.h"
|
|
#include "ut0crc32.h"
|
|
#include "ut0rnd.h"
|
|
|
|
#ifndef UNIV_INNOCHECKSUM
|
|
#include "srv0srv.h"
|
|
#endif /* !UNIV_INNOCHECKSUM */
|
|
|
|
/** Calculate the CRC32 checksum of a page. The value is stored to the page
|
|
when it is written to a file and also checked for a match when reading from
|
|
the file. Note that we must be careful to calculate the same value on all
|
|
architectures.
|
|
@param[in] page buffer page (srv_page_size bytes)
|
|
@return CRC-32C */
|
|
uint32_t buf_calc_page_crc32(const byte* page)
|
|
{
|
|
/* Note: innodb_checksum_algorithm=crc32 could and should have
|
|
included the entire page in the checksum, and CRC-32 values
|
|
should be combined with the CRC-32 function, not with
|
|
exclusive OR. We stick to the current algorithm in order to
|
|
remain compatible with old data files. */
|
|
return ut_crc32(page + FIL_PAGE_OFFSET,
|
|
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION
|
|
- FIL_PAGE_OFFSET)
|
|
^ ut_crc32(page + FIL_PAGE_DATA,
|
|
srv_page_size
|
|
- (FIL_PAGE_DATA + FIL_PAGE_END_LSN_OLD_CHKSUM));
|
|
}
|
|
|
|
#ifndef UNIV_INNOCHECKSUM
|
|
/** Calculate a checksum which is stored to the page when it is written
|
|
to a file. Note that we must be careful to calculate the same value on
|
|
32-bit and 64-bit architectures.
|
|
@param[in] page file page (srv_page_size bytes)
|
|
@return checksum */
|
|
uint32_t
|
|
buf_calc_page_new_checksum(const byte* page)
|
|
{
|
|
ulint checksum;
|
|
|
|
/* Since the field FIL_PAGE_FILE_FLUSH_LSN, and in versions <= 4.1.x
|
|
FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID, are written outside the buffer pool
|
|
to the first pages of data files, we have to skip them in the page
|
|
checksum calculation.
|
|
We must also skip the field FIL_PAGE_SPACE_OR_CHKSUM where the
|
|
checksum is stored, and also the last 8 bytes of page because
|
|
there we store the old formula checksum. */
|
|
|
|
checksum = ut_fold_binary(page + FIL_PAGE_OFFSET,
|
|
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION
|
|
- FIL_PAGE_OFFSET)
|
|
+ ut_fold_binary(page + FIL_PAGE_DATA,
|
|
srv_page_size - FIL_PAGE_DATA
|
|
- FIL_PAGE_END_LSN_OLD_CHKSUM);
|
|
return(static_cast<uint32_t>(checksum));
|
|
}
|
|
|
|
/** In MySQL before 4.0.14 or 4.1.1 there was an InnoDB bug that
|
|
the checksum only looked at the first few bytes of the page.
|
|
This calculates that old checksum.
|
|
NOTE: we must first store the new formula checksum to
|
|
FIL_PAGE_SPACE_OR_CHKSUM before calculating and storing this old checksum
|
|
because this takes that field as an input!
|
|
@param[in] page file page (srv_page_size bytes)
|
|
@return checksum */
|
|
uint32_t
|
|
buf_calc_page_old_checksum(const byte* page)
|
|
{
|
|
return(static_cast<uint32_t>
|
|
(ut_fold_binary(page, FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION)));
|
|
}
|
|
#endif /* !UNIV_INNOCHECKSUM */
|