mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-30 18:41:56 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/storage/innobase/include
History
Thirunarayanan Balathandayuthapani c0f47a4a58 MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 
MariaDB data-at-rest encryption (innodb_encrypt_tables)
had repurposed the same unused data field that was repurposed
in MySQL 5.7 (and MariaDB 10.2) for the Split Sequence Number (SSN)
field of SPATIAL INDEX. Because of this, MariaDB was unable to
support encryption on SPATIAL INDEX pages.

Furthermore, InnoDB page checksums skipped some bytes, and there
are multiple variations and checksum algorithms. By default,
InnoDB accepts all variations of all algorithms that ever existed.
This unnecessarily weakens the page checksums.

We hereby introduce two more innodb_checksum_algorithm variants
(full_crc32, strict_full_crc32) that are special in a way:
When either setting is active, newly created data files will
carry a flag (fil_space_t::full_crc32()) that indicates that
all pages of the file will use a full CRC-32C checksum over the
entire page contents (excluding the bytes where the checksum
is stored, at the very end of the page). Such files will always
use that checksum, no matter what the parameter
innodb_checksum_algorithm is assigned to.

For old files, the old checksum algorithms will continue to be
used. The value strict_full_crc32 will be equivalent to strict_crc32
and the value full_crc32 will be equivalent to crc32.

ROW_FORMAT=COMPRESSED tables will only use the old format.
These tables do not support new features, such as larger
innodb_page_size or instant ADD/DROP COLUMN. They may be
deprecated in the future. We do not want an unnecessary
file format change for them.

The new full_crc32() format also cleans up the MariaDB tablespace
flags. We will reserve flags to store the page_compressed
compression algorithm, and to store the compressed payload length,
so that checksum can be computed over the compressed (and
possibly encrypted) stream and can be validated without
decrypting or decompressing the page.

In the full_crc32 format, there no longer are separate before-encryption
and after-encryption checksums for pages. The single checksum is
computed on the page contents that is written to the file.

We do not make the new algorithm the default for two reasons.
First, MariaDB 10.4.2 was a beta release, and the default values
of parameters should not change after beta. Second, we did not
yet implement the full_crc32 format for page_compressed pages.
This will be fixed in MDEV-18644.

This is joint work with Marko Mäkelä.
2019-02-19 18:50:19 +02:00
..
btr0btr.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
btr0btr.ic MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
btr0bulk.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
btr0cur.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
btr0cur.ic Fix many -Wunused-parameter 2018-05-01 16:52:19 +03:00
btr0defragment.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
btr0pcur.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
btr0pcur.ic Merge 10.2 into 10.3 2018-07-06 22:18:35 +03:00
btr0scrub.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
btr0sea.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
btr0sea.ic Merge 10.2 into 10.3 2018-07-23 18:56:52 +03:00
btr0types.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
buf0buddy.h More InnoDB preprocessor cleanup 2018-11-30 21:50:53 +02:00
buf0buddy.ic More InnoDB preprocessor cleanup 2018-11-30 21:50:53 +02:00
buf0buf.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
buf0buf.ic MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-07 19:45:50 +04:00
buf0checksum.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
buf0dblwr.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
buf0dump.h
buf0flu.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
buf0flu.ic Remove code duplication around buf_pool->flush_rbt 2019-01-11 15:47:35 +02:00
buf0lru.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
buf0rea.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
buf0types.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
data0data.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
data0data.ic MDEV-11369 Instant ADD COLUMN for InnoDB 2017-10-06 09:50:10 +03:00
data0type.h MDEV-15564: Fix bool/ibool type mismatch 2019-02-15 08:15:46 +02:00
data0type.ic MDEV-18627 Wrong result after instant size change of integer 2019-02-18 18:30:01 +02:00
data0types.h
db0err.h MDEV-13564: Remove old crash-upgrade logic in 10.4 2018-09-11 21:32:15 +03:00
dict0boot.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
dict0boot.ic Use compile_time_assert() in InnoDB 2018-04-30 18:22:52 +03:00
dict0crea.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
dict0crea.ic
dict0defrag_bg.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
dict0dict.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
dict0dict.ic MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
dict0load.h Merge 10.2 into 10.3 2018-11-19 18:45:53 +02:00
dict0mem.h MDEV-18627 Wrong result after instant size change of integer 2019-02-18 18:30:01 +02:00
dict0mem.ic MDEV-12266: Remove dict_index_t::table_name 2018-03-29 20:47:37 +03:00
dict0pagecompress.h
dict0pagecompress.ic
dict0priv.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
dict0priv.ic MDEV-17794 Do not assign persistent ID for temporary tables 2018-11-22 15:42:52 +02:00
dict0stats.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
dict0stats.ic Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
dict0stats_bg.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
dict0types.h Fix most -Wsign-conversion in InnoDB 2018-04-28 20:45:45 +03:00
dyn0buf.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
dyn0types.h
eval0eval.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
eval0eval.ic
eval0proc.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
eval0proc.ic
fil0crypt.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fil0crypt.ic
fil0fil.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fil0fil.ic MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fil0pagecompress.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fsp0file.h Merge 10.2 into 10.3 2019-02-04 15:12:14 +02:00
fsp0fsp.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fsp0fsp.ic MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
fsp0pagecompress.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fsp0pagecompress.ic MDEV-13103 Deal with page_compressed page corruption 2018-06-14 14:23:01 +03:00
fsp0space.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fsp0sysspace.h Merge 10.2 into 10.3 2018-11-19 18:45:53 +02:00
fsp0types.h MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 2019-02-19 18:50:19 +02:00
fts0ast.h Remove many redundant #include from InnoDB 2018-11-19 11:42:14 +02:00
fts0blex.h
fts0fts.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
fts0opt.h
fts0pars.h
fts0plugin.h Remove many redundant #include from InnoDB 2018-11-19 11:42:14 +02:00
fts0priv.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
fts0priv.ic
fts0tlex.h
fts0tokenize.h Fix most -Wsign-conversion in InnoDB 2018-04-28 20:45:45 +03:00
fts0types.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
fts0types.ic Merge 10.2 into 10.3 2018-12-12 13:46:06 +02:00
fts0vlc.ic
fut0fut.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
fut0lst.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
fut0lst.ic MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
gis0geo.h
gis0rtree.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
gis0rtree.ic MDEV-11369 Instant ADD COLUMN for InnoDB 2017-10-06 09:50:10 +03:00
gis0type.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
ha0ha.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
ha0ha.ic Reduce the number of rw_lock_own() calls 2018-07-23 17:49:01 +03:00
ha0storage.h
ha0storage.ic Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
ha_prototypes.h InnoDB review fixes 2019-02-05 21:51:35 +02:00
handler0alter.h MDEV-13359 Enable ALTER TABLE...ALGORITHM=INPLACE for compressed columns 2017-09-01 16:36:11 +03:00
hash0hash.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
hash0hash.ic
ib0mutex.h MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-09 20:54:04 +04:00
ibuf0ibuf.h Do not write MLOG_IBUF_BITMAP_INIT 2019-02-08 19:10:33 +02:00
ibuf0ibuf.ic MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
ibuf0types.h
lock0iter.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
lock0lock.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
lock0lock.ic Merge branch '10.2' into 10.3 2018-03-28 17:31:57 +02:00
lock0prdt.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
lock0priv.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
lock0priv.ic Rename lock_pool_t to lock_list 2018-08-16 05:53:50 +03:00
lock0types.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
log0crypt.h MDEV-12041: innodb_encrypt_log key rotation 2018-08-13 16:04:37 +03:00
log0log.h Merge 10.3 into 10.4 2019-01-15 12:30:29 +02:00
log0log.ic MDEV-12041: innodb_encrypt_log key rotation 2018-08-13 16:04:37 +03:00
log0recv.h Merge 10.3 into 10.4 2018-11-19 20:22:33 +02:00
log0types.h
mach0data.h Replace dict_instant_t::non_pk_col_map with field_map 2018-12-13 22:23:49 +02:00
mach0data.ic MDEV-11939: innochecksum mistakes a file for an encrypted one (page 0 invalid) 2017-08-08 09:41:09 +03:00
mem0mem.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
mem0mem.ic MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
mtr0log.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
mtr0log.ic Replace univ_page_size and UNIV_PAGE_SIZE 2018-04-28 20:45:45 +03:00
mtr0mtr.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
mtr0mtr.ic MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-07 19:45:50 +04:00
mtr0types.h MDEV-15528: Introduce MLOG_INIT_FREE_PAGE 2019-02-08 19:16:32 +02:00
os0api.h
os0event.h Fix many -Wunused-parameter 2018-05-01 16:52:19 +03:00
os0file.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
os0file.ic Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
os0once.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
os0proc.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
os0thread.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
page0cur.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
page0cur.ic More InnoDB preprocessor cleanup 2018-11-30 21:50:53 +02:00
page0page.h Merge 10.3 into 10.4 2018-12-13 22:05:34 +02:00
page0page.ic More InnoDB preprocessor cleanup 2018-11-30 21:50:53 +02:00
page0types.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
page0zip.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
page0zip.ic MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
pars0grm.h
pars0opt.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
pars0pars.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
pars0sym.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
pars0types.h
que0que.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
que0que.ic Dead code removal: sess_t 2018-02-15 15:00:46 +00:00
que0types.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
read0types.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-29 14:02:15 +04:00
rem0cmp.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
rem0cmp.ic Remove many redundant #include from InnoDB 2018-11-19 11:42:14 +02:00
rem0rec.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
rem0rec.ic MDEV-17794 Do not assign persistent ID for temporary tables 2018-11-22 15:42:52 +02:00
rem0types.h
row0ext.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
row0ext.ic
row0ftsort.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
row0import.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
row0ins.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
row0log.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
row0log.ic
row0merge.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
row0mysql.h InnoDB review fixes 2019-02-05 21:51:35 +02:00
row0purge.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
row0quiesce.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
row0row.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
row0row.ic MDEV-17750: Remove dict_index_get_sys_col_pos() 2018-11-16 20:37:06 +02:00
row0sel.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
row0sel.ic
row0types.h MDEV-15855 cleanup: Privatize purge_vcol_info_t 2018-07-23 18:31:42 +03:00
row0uins.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
row0umod.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
row0undo.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
row0upd.h Merge 10.3 into 10.4 2019-01-24 22:42:35 +02:00
row0upd.ic MDEV-17750: Remove dict_index_get_sys_col_pos() 2018-11-16 20:37:06 +02:00
row0vers.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
srv0conc.h MDEV-12218 Clean up InnoDB parameter validation 2018-04-29 09:41:42 +03:00
srv0mon.h InnoDB review fixes 2019-02-05 21:51:35 +02:00
srv0mon.ic
srv0srv.h MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-07 19:45:50 +04:00
srv0start.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
sync0arr.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
sync0arr.ic ib_counter_t code simplified without functional changes 2018-10-11 22:55:52 +03:00
sync0debug.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
sync0policy.h MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-09 20:54:04 +04:00
sync0rw.h MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-09 20:54:04 +04:00
sync0rw.ic MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
sync0sync.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
sync0types.h MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-09 20:54:04 +04:00
trx0i_s.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
trx0purge.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-28 13:24:11 +04:00
trx0purge.ic Rename the purge_sys_t iterators 2018-02-21 19:16:47 +02:00
trx0rec.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
trx0rec.ic MDEV-14717 RENAME TABLE in InnoDB is not crash-safe 2018-09-07 22:10:02 +03:00
trx0roll.h Merge 10.3 into 10.4 2018-11-30 11:05:46 +02:00
trx0rseg.h Replace univ_page_size and UNIV_PAGE_SIZE 2018-04-28 20:45:45 +03:00
trx0rseg.ic MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
trx0sys.h MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
trx0trx.h Galera4 2019-01-23 15:30:00 +04:00
trx0trx.ic MDEV-16136: Prevent wrong reuse in trx_reference() 2018-08-16 05:54:11 +03:00
trx0types.h Merge 10.2 into 10.3 2018-11-19 18:45:53 +02:00
trx0undo.h MDEV-17794 Do not assign persistent ID for temporary tables 2018-11-22 15:42:52 +02:00
trx0undo.ic MDEV-18493 Remove page_size_t 2019-02-07 12:21:35 +02:00
trx0xa.h
univ.i Merge 10.3 into 10.4 2019-01-24 22:42:35 +02:00
ut0byte.h
ut0byte.ic Use compile_time_assert() in InnoDB 2018-04-30 18:22:52 +03:00
ut0counter.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-27 22:46:38 +04:00
ut0crc32.h MDEV-17958: Remove WITH_INNODB_BUG_ENDIAN_CRC32 2018-12-13 22:19:09 +02:00
ut0dbg.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
ut0list.h
ut0list.ic
ut0lst.h Fix many -Wunused-parameter 2018-05-01 16:52:19 +03:00
ut0mem.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
ut0mem.ic
ut0mutex.h MDEV-17441 - InnoDB transition to C++11 atomics 2019-01-09 20:54:04 +04:00
ut0new.h Merge 10.2 into 10.3 2018-11-19 18:45:53 +02:00
ut0pool.h Merge 10.2 into 10.3 2018-08-16 08:54:58 +03:00
ut0rbt.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
ut0rnd.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
ut0rnd.ic MDEV-14482 - Cache line contention on ut_rnd_ulint_counter() 2018-01-26 10:25:33 +04:00
ut0sort.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
ut0stage.h Merge 10.2 into 10.3 2018-11-30 09:38:56 +02:00
ut0timer.h Merge 10.1 into 10.2 2018-06-06 11:25:33 +03:00
ut0timer.ic Remove dead code that was added in MDEV-5834 2018-06-05 11:39:47 +03:00
ut0ut.h MDEV-17441 - InnoDB transition to C++11 atomics 2018-12-29 14:02:15 +04:00
ut0ut.ic
ut0vec.h Remove some unnecessary InnoDB #include 2018-11-29 12:53:44 +02:00
ut0vec.ic Remove many redundant #include from InnoDB 2018-11-19 11:42:14 +02:00
ut0wqueue.h