mariadb/storage/innobase/include/fsp0space.h
Thirunarayanan Balathandayuthapani c0f47a4a58 MDEV-12026: Implement innodb_checksum_algorithm=full_crc32
MariaDB data-at-rest encryption (innodb_encrypt_tables)
had repurposed the same unused data field that was repurposed
in MySQL 5.7 (and MariaDB 10.2) for the Split Sequence Number (SSN)
field of SPATIAL INDEX. Because of this, MariaDB was unable to
support encryption on SPATIAL INDEX pages.

Furthermore, InnoDB page checksums skipped some bytes, and there
are multiple variations and checksum algorithms. By default,
InnoDB accepts all variations of all algorithms that ever existed.
This unnecessarily weakens the page checksums.

We hereby introduce two more innodb_checksum_algorithm variants
(full_crc32, strict_full_crc32) that are special in a way:
When either setting is active, newly created data files will
carry a flag (fil_space_t::full_crc32()) that indicates that
all pages of the file will use a full CRC-32C checksum over the
entire page contents (excluding the bytes where the checksum
is stored, at the very end of the page). Such files will always
use that checksum, no matter what the parameter
innodb_checksum_algorithm is assigned to.

For old files, the old checksum algorithms will continue to be
used. The value strict_full_crc32 will be equivalent to strict_crc32
and the value full_crc32 will be equivalent to crc32.

ROW_FORMAT=COMPRESSED tables will only use the old format.
These tables do not support new features, such as larger
innodb_page_size or instant ADD/DROP COLUMN. They may be
deprecated in the future. We do not want an unnecessary
file format change for them.

The new full_crc32() format also cleans up the MariaDB tablespace
flags. We will reserve flags to store the page_compressed
compression algorithm, and to store the compressed payload length,
so that checksum can be computed over the compressed (and
possibly encrypted) stream and can be validated without
decrypting or decompressing the page.

In the full_crc32 format, there no longer are separate before-encryption
and after-encryption checksums for pages. The single checksum is
computed on the page contents that is written to the file.

We do not make the new algorithm the default for two reasons.
First, MariaDB 10.4.2 was a beta release, and the default values
of parameters should not change after beta. Second, we did not
yet implement the full_crc32 format for page_compressed pages.
This will be fixed in MDEV-18644.

This is joint work with Marko Mäkelä.
2019-02-19 18:50:19 +02:00

242 lines
6.3 KiB
C++

/*****************************************************************************
Copyright (c) 2013, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2017, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Suite 500, Boston, MA 02110-1335 USA
*****************************************************************************/
/**************************************************//**
@file include/fsp0space.h
Shared tablespace interface
Created 2013-7-26 by Kevin Lewis
*******************************************************/
#ifndef fsp0space_h
#define fsp0space_h
#include "fsp0file.h"
#include "fsp0fsp.h"
#include "fsp0types.h"
#include <vector>
/** Data structure that contains the information about shared tablespaces.
Currently this can be the system tablespace or a temporary table tablespace */
class Tablespace {
public:
typedef std::vector<Datafile, ut_allocator<Datafile> > files_t;
/** Data file information - each Datafile can be accessed globally */
files_t m_files;
/** Data file iterator */
typedef files_t::iterator iterator;
/** Data file iterator */
typedef files_t::const_iterator const_iterator;
Tablespace()
:
m_files(),
m_name(),
m_space_id(ULINT_UNDEFINED),
m_path(),
m_flags(),
m_ignore_read_only(false)
{
/* No op */
}
virtual ~Tablespace()
{
shutdown();
ut_ad(m_files.empty());
ut_ad(m_space_id == ULINT_UNDEFINED);
}
// Disable copying
Tablespace(const Tablespace&);
Tablespace& operator=(const Tablespace&);
/** Data file iterator */
const_iterator begin() const { return m_files.begin(); }
/** Data file iterator */
const_iterator end() const { return m_files.end(); }
/** Data file iterator */
iterator begin() { return m_files.begin(); }
/** Data file iterator */
iterator end() { return m_files.end(); }
void set_name(const char* name) { m_name = name; }
const char* name() const { return m_name; }
/** Set tablespace path and filename members.
@param[in] path where tablespace file(s) resides
@param[in] len length of the file path */
void set_path(const char* path, size_t len)
{
ut_ad(m_path == NULL);
m_path = mem_strdupl(path, len);
ut_ad(m_path != NULL);
os_normalize_path(m_path);
}
/** Set tablespace path and filename members.
@param[in] path where tablespace file(s) resides */
void set_path(const char* path)
{
set_path(path, strlen(path));
}
/** Get tablespace path
@return tablespace path */
const char* path() const
{
return(m_path);
}
/** Set the space id of the tablespace
@param[in] space_id tablespace ID to set */
void set_space_id(ulint space_id)
{
ut_ad(m_space_id == ULINT_UNDEFINED);
m_space_id = space_id;
}
/** Get the space id of the tablespace
@return m_space_id space id of the tablespace */
ulint space_id() const
{
return(m_space_id);
}
/** Set the tablespace flags
@param[in] fsp_flags tablespace flags */
void set_flags(ulint fsp_flags)
{
ut_ad(fil_space_t::is_valid_flags(fsp_flags, false));
m_flags = fsp_flags;
}
/** Get the tablespace flags
@return m_flags tablespace flags */
ulint flags() const
{
return(m_flags);
}
/** Get the tablespace encryption mode
@return m_mode tablespace encryption mode */
fil_encryption_t encryption_mode() const
{
return (m_mode);
}
/** Get the tablespace encryption key_id
@return m_key_id tablespace encryption key_id */
uint32_t key_id() const
{
return (m_key_id);
}
/** Set Ignore Read Only Status for tablespace.
@param[in] read_only_status read only status indicator */
void set_ignore_read_only(bool read_only_status)
{
m_ignore_read_only = read_only_status;
}
/** Free the memory allocated by the Tablespace object */
void shutdown();
/** @return the sum of the file sizes of each Datafile */
ulint get_sum_of_sizes() const
{
ulint sum = 0;
for (const_iterator it = begin(); it != end(); ++it) {
sum += it->m_size;
}
return(sum);
}
/** Open or Create the data files if they do not exist.
@param[in] is_temp whether this is a temporary tablespace
@return DB_SUCCESS or error code */
dberr_t open_or_create(bool is_temp)
MY_ATTRIBUTE((warn_unused_result));
/** Delete all the data files. */
void delete_files();
/** Check if two tablespaces have common data file names.
@param[in] other_space Tablespace to check against this.
@return true if they have the same data filenames and paths */
bool intersection(const Tablespace* other_space);
/** Use the ADD DATAFILE path to create a Datafile object and add
it to the front of m_files. Parse the datafile path into a path
and a basename with extension 'ibd'. This datafile_path provided
may be an absolute or relative path, but it must end with the
extension .ibd and have a basename of at least 1 byte.
Set tablespace m_path member and add a Datafile with the filename.
@param[in] datafile_path full path of the tablespace file. */
dberr_t add_datafile(
const char* datafile_path);
/* Return a pointer to the first Datafile for this Tablespace
@return pointer to the first Datafile for this Tablespace*/
Datafile* first_datafile()
{
ut_a(!m_files.empty());
return(&m_files.front());
}
private:
/**
@param[in] filename Name to lookup in the data files.
@return true if the filename exists in the data files */
bool find(const char* filename) const;
/** Note that the data file was found.
@param[in] file data file object */
void file_found(Datafile& file);
/* DATA MEMBERS */
/** Name of the tablespace. */
const char* m_name;
/** Tablespace ID */
ulint m_space_id;
/** Path where tablespace files will reside, not including a filename.*/
char* m_path;
/** Tablespace flags */
ulint m_flags;
/** Encryption mode and key_id */
fil_encryption_t m_mode;
uint32_t m_key_id;
protected:
/** Ignore server read only configuration for this tablespace. */
bool m_ignore_read_only;
};
#endif /* fsp0space_h */