- Removed duplicate words, like "the the" and "to to"
- Removed duplicate lines (one double sort line found in mysql.cc)
- Fixed some typos found while searching for duplicate words.
Command used to find duplicate words:
egrep -rI "\s([a-zA-Z]+)\s+\1\s" | grep -v param
Thanks to Artjoms Rimdjonoks for the command and pointing out the
spelling errors.
These changes was done as part of fixing
MDEV-36858 MariaDB MyISAM secondary indexes silently break for
tables > 10B rows
Changes done in myisamchk:
- Tables that are checked are opened in readonly mode if --force is not
used.
- *.MYD files will be opened in readonly mode for repair if --quick
is used.
- Added information about check progress if --verbose is used.
- Output information about repaired/checked rows every 10000 rows instead
of every 1000 rows. Note that this also affects aria_chk
- Store open file mode in share->index_mode and share->data_mode instead
of in share->mode.
- Added new option --keys-active= as a simpler version of keys-used.
- Changed output for "myisamchk -dvv" to get nicer output for tables
with 10 billion rows.
MSAN has been updated since 2022 when this macro was added
and as such the working around MSAN's deficient understanding
of the fstat/stat syscall behaviour at the time is no longer
required.
As an effective no-op a straight removal is sufficient.
When table2myisam() prepares recinfo structures BIT field was skipped
because pack_length_in_rec() returns 0. Instead of BIT field
DB_ROW_HASH_1 field was taken into recinfo structure and its length
was added to reclength. This is wrong because not stored fields must
not be prepared as record columns (MI_COLUMNDEF) in storage
layer. 0-length fields are prepared in "reserve space for null bits"
branch.
The problem only occurs with tables where there is no data for the
main record outside of the null bits.
The fix updates minpos condition so we avoid fields after
stored_rec_length and these are not stored by
definition. share->reclength already includes not stored lengths from
CREATE TABLE so we cannot use it as minpos starting point.
In Aria there is no "reserve space for null bits" and it does not
create column definition for BIT. Also there is no
setup_vcols_for_repair() to reproduce the issue. But nonetheless it
creates column definition for not stored fields redundantly, so we
patch table2maria() as well. The test case for Aria tries to
demonstrate BIT field works, it does not reproduce any issues (as
redundant column definition for not stored field seem to not cause any
problems).
in _ma_unique_hash, skip_trailing_space, my_hash_sort_mb_nopad_bin and my_strnncollsp_utf8mb4_bin
UBSAN detected the nullptr-with-offset in a few places
when handling empty blobs.
Fix:
- Adding DBUG_ASSERT(source_string) into all hash_sort() implementations
to catch this problem in non-UBSAN debug builds.
- Fixing mi_unique_hash(), mi_unique_comp(),
_ma_unique_hash(), _ma_unique_comp() to replace NULL pointer to
an empty string ponter..
Note, we should also add DBUG_ASSERT(source_string != NULL) into
all implementations of strnncoll*(). But I'm afraid the patch
is going to be too long and too dangerous for 10.5.
* Innobase `os0file.cc`: use `PRIu64` over `llu`
* These came after I prepared #3485.
* MyISAM `mi_check.c`: in impossible block length warning
* I missed this one in #3485 (and #3360 too?).
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
Partial commit of the greater MDEV-34348 scope.
MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict
The functions queue_compare, qsort2_cmp, and qsort_cmp2
all had similar interfaces, and were used interchangable
and unsafely cast to one another.
This patch consolidates the functions all into the
qsort_cmp2 interface.
Reviewed By:
============
Marko Mäkelä <marko.makela@mariadb.com>
heap-buffer-overflow in _mi_put_key_in_record
Rec buffer size depends on vreclength like this:
length= MY_MAX(length, info->s->vreclength);
The problem is rec buffer is allocated before vreclength is
calculated. The fix reallocates rec buffer if vreclength changed.
1. Rec buffer allocated
f0 mi_alloc_rec_buff (...) at ../src/storage/myisam/mi_open.c:738
f1 0x00005f4928244516 in mi_open (...) at ../src/storage/myisam/mi_open.c:671
f2 0x00005f4928210b98 in ha_myisam::open (...)
at ../src/storage/myisam/ha_myisam.cc:847
f3 0x00005f49273aba41 in handler::ha_open (...) at ../src/sql/handler.cc:3105
f4 0x00005f4927995a65 in open_table_from_share (...)
at ../src/sql/table.cc:4320
f5 0x00005f492769f084 in open_table (...) at ../src/sql/sql_base.cc:2024
f6 0x00005f49276a3ea9 in open_and_process_table (...)
at ../src/sql/sql_base.cc:3819
f7 0x00005f49276a29b8 in open_tables (...) at ../src/sql/sql_base.cc:4303
f8 0x00005f49276a6f3f in open_and_lock_tables (...)
at ../src/sql/sql_base.cc:5250
f9 0x00005f49275162de in open_and_lock_tables (...)
at ../src/sql/sql_base.h:509
f10 0x00005f4927a30d7a in open_only_one_table (...)
at ../src/sql/sql_admin.cc:412
f11 0x00005f4927a2c0c2 in mysql_admin_table (...)
at ../src/sql/sql_admin.cc:603
f12 0x00005f4927a2fda8 in Sql_cmd_optimize_table::execute (...)
at ../src/sql/sql_admin.cc:1517
f13 0x00005f49278102e3 in mysql_execute_command (...)
at ../src/sql/sql_parse.cc:6180
f14 0x00005f49278012d7 in mysql_parse (...) at ../src/sql/sql_parse.cc:8236
2. vreclength calculated
f0 ha_myisam::setup_vcols_for_repair (...)
at ../src/storage/myisam/ha_myisam.cc:1002
f1 0x00005f49282138b4 in ha_myisam::optimize (...)
at ../src/storage/myisam/ha_myisam.cc:1250
f2 0x00005f49273b4961 in handler::ha_optimize (...)
at ../src/sql/handler.cc:4896
f3 0x00005f4927a2d254 in mysql_admin_table (...)
at ../src/sql/sql_admin.cc:875
f4 0x00005f4927a2fda8 in Sql_cmd_optimize_table::execute (...)
at ../src/sql/sql_admin.cc:1517
f5 0x00005f49278102e3 in mysql_execute_command (...)
at ../src/sql/sql_parse.cc:6180
f6 0x00005f49278012d7 in mysql_parse (...) at ../src/sql/sql_parse.cc:8236
FYI backtrace was done with
set print frame-info location
set print frame-arguments presence
set width 80
Assertion `table->field[0]->ptr >= table->record[0] &&
table->field[0]->ptr <= table->record[0] + table->s->reclength' failed in
handler::assert_icp_limitations.
table->move_fields has some limitations:
1. It cannot be used in cascade
2. It should always have a restoring pair.
Rule 1 is covered by assertions in handler::assert_icp_limitations
and handler::ptr_in_record (commit 30894fe9a9).
Rule 2 should be manually maintained with care. Hopefully, the rule 1 assertions
may sometimes help as well.
In ha_myisam::repair, both rules are broken. table->move_fields is used
asymmetrically there: it is set on every param->fix_record call
(i.e. in compute_vcols) but is restored only once, in the end of repair.
The reason to updating field ptr's for every call is that compute_vcols can
(supposedly) be called in parallel, that is, with the same table, but different
records.
The condition to "unmove" the pointers in ha_myisam::restore_vcos_after_repair
is incorrect, when stored vcols are available, and myisam stores a VIRTUAL field
if it's the only field in the table (the record cannot be of zero length).
This patch solves the problem by "unmoving" the pointers symmetrically, in
compute_vcols. That is, both rules will be preserved maintained.
