strict aliasing violations.
Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.
As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.
The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.
client/mysql.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysql_upgrade.c:
Remove now-unnecessary casts.
client/mysqladmin.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysqlbinlog.cc:
Remove now-unnecessary casts.
client/mysqlcheck.c:
Remove now-unnecessary casts.
client/mysqldump.c:
Remove now-unnecessary casts.
client/mysqlimport.c:
Remove now-unnecessary casts.
client/mysqlshow.c:
Remove now-unnecessary casts.
client/mysqlslap.c:
Remove now-unnecessary casts.
client/mysqltest.cc:
Remove now-unnecessary casts.
extra/comp_err.c:
Remove now-unnecessary casts.
extra/my_print_defaults.c:
Remove now-unnecessary casts.
Break up large strings.
extra/mysql_waitpid.c:
Remove now-unnecessary casts.
extra/perror.c:
Remove now-unnecessary casts.
extra/resolve_stack_dump.c:
Remove now-unnecessary casts.
extra/resolveip.c:
Remove now-unnecessary casts.
include/my_getopt.h:
Use a void pointer type as the opaque type to avoid problems with type
incompatibility -- GCC issues warnings when the type name is not type
compatible with a operand. As a side bonus, a explicit cast won't be
necessary anymore.
include/sslopt-longopts.h:
Remove now-unnecessary casts.
Break up large strings.
mysys/my_getopt.c:
Update opaque type and introduce a type definition for the
argument to my_getopt_register_get_addr.
server-tools/instance-manager/options.cc:
Remove now-unnecessary casts.
sql/mysqld.cc:
Remove now-unnecessary casts.
Break up large strings.
Update mysql_getopt_value prototype (the old prototype
was different from the definition anyway).
sql/sql_plugin.cc:
The type of a pointer to a function must be compatible with the
pointed-to function type, otherwise the behavior is undefined.
sql/table.cc:
The variable buf pointer to pointer to pointer to constant char
could improperly alias a incompatible type in call to fix_type_
pointers. Since this was actually dead code, it is simply removed.
sql/unireg.cc:
Remove call to get_form_pos. The code creates a new FRM file which
is always truncated and writes the form position as 0. Hence, no
need to retrieve it, we now for sure it is 0.
storage/archive/archive_reader.c:
Remove now-unnecessary casts.
storage/myisam/ft_nlq_search.c:
Read weight directly from the buffer.
storage/myisam/fulltext.h:
Add explanation about the type duality of a key buffer.
Add accessor macro to retrieve a FT float value.
storage/myisam/mi_test1.c:
Remove now-unnecessary casts.
storage/myisam/myisam_ftdump.c:
Read weight directly from the buffer.
storage/myisam/myisamchk.c:
Remove now-unnecessary casts.
storage/myisam/myisamlog.c:
A pointer to char was used to alias a pointer to pointer to
unsigned char, thus violating strict aliasing rules.
storage/myisam/myisampack.c:
Remove now-unnecessary casts.
strings/decimal.c:
Remove aliasing violation, printing the value is enough for
debugging purposes.
tests/mysql_client_test.c:
Remove now-unnecessary casts.
information_schema query.
Porting a fix done for a feature tree by Ingo Struewing.
No test case since the crash only occurs under a stress
load.
storage/myisam/mi_dbug.c:
Iterate over the list of MyISAM shares and access the MyISAM
share internals under a lock.
Add -Wall to gcc/g++
Fix most warnings reported in dbg and opt mode.
cmd-line-utils/libedit/filecomplete.c:
Remove unused auto variables.
configure.cmake:
Add -Wall to gcc.
extra/comp_err.c:
Cast to correct type.
extra/perror.c:
Fix segfault (but warnings about deprecated features remain)
extra/yassl/taocrypt/include/runtime.hpp:
Comparing two literals was reported as undefined behaviour.
include/my_global.h:
Add a template for aligning character buffers.
mysys/lf_alloc-pin.c:
Initialize pointer.
sql/mysqld.cc:
Use UNINIT_VAR rather than LINT_INIT.
sql/partition_info.cc:
Use UNINIT_VAR rather than LINT_INIT.
sql/rpl_handler.cc:
Use char[] rather than unsigned long[] array for placement buffer.
sql/spatial.cc:
Use char[] rather than unsigned void*[] array for placement buffer.
sql/spatial.h:
Use char[] rather than unsigned void*[] array for placement buffer.
sql/sql_partition.cc:
Initialize auto variable.
sql/sql_table.cc:
Initialize auto variables.
Add parens around assignment within if()
sql/sys_vars.cc:
Use UNINIT_VAR.
storage/innobase/os/os0file.c:
Init first slot in auto variable.
storage/myisam/mi_create.c:
Use UNINIT_VAR rather than LINT_INIT.
storage/myisam/mi_open.c:
Remove (wrong) casting.
storage/myisam/mi_page.c:
Remove (wrong) casting.
storage/myisam/mi_search.c:
Cast to uchar* rather than char*.
strings/ctype-ucs2.c:
Use UNINIT_VAR rather than LINT_INIT.
Add (uchar*) casting.
(make relies GNU extentions). The patch was partially
backport from 6.0.
Original comment:
bug#30708: make relies GNU extensions. Now that we no longer use
BitKeeper we can safely remove the SCCS handling with no loss of
functionality.
even if myisam-recover is OFF
The problem was that a corrupted MyISAM table was auto repaired
even if the myisam_recover_options server variable (or the
myisam_recover option) was set to OFF.
The reason was that the auto_repair() function, which is supposed
to say if auto repair is to be used, did not use the server variable
setting correctly. This bug was a regression introduced by WL#4738.
This patch fixes the problem by making sure auto_repair() returns
FALSE if myisam_recover_options is set to OFF.
Test case added to myisam.test.
Conflicts:
Text conflict in mysql-test/r/func_str.result
Text conflict in mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_32.result
Text conflict in mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_64.result
Text conflict in mysql-test/t/func_str.test
Text conflict in sql/mysqld.cc
Text conflict in sql/protocol.cc
Text conflict in storage/myisam/mi_open.c
data and index files
It was possible if DATA/INDEX DIRECTORY is pointing to
symlinked MySQL data home directory.
Do not allow to drop data/index files implicitly symlinked
to data home directory. For such tables remove symlink only.
mysql-test/r/symlink.result:
A test case for BUG#40980.
mysql-test/t/symlink.test:
A test case for BUG#40980.
storage/myisam/mi_delete_table.c:
Do not allow to drop data/index files implicitly symlinked
to data home directory. For such tables remove symlink only.
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
Repairing MyISAM table with fulltext indexes and low
myisam_sort_buffer_size may crash the server.
Estimation of number of index entries was done incorrectly,
causing further assertion failure or server crash.
Docs note: min value for myisam_sort_buffer_size has been
changed from 4 to 4096.
mysql-test/r/fulltext.result:
A test case for BUG#51866.
mysql-test/r/myisam.result:
Min value for myisam_sort_buffer_size is 4096.
mysql-test/r/variables.result:
Min value for myisam_sort_buffer_size is 4096.
mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic_32.result:
Min value for myisam_sort_buffer_size is 4096.
mysql-test/t/fulltext.test:
A test case for BUG#51866.
sql/mysqld.cc:
Min value for myisam_sort_buffer_size is 4096.
storage/myisam/mi_check.c:
When estimating number of index entries for external
fulltext parser, take into account that key_length may
be bigger than myisam_sort_buffer_size. Reuse logic
from _create_index_by_sort(): force MIN_SORT_BUFFER to
be min value for myisam_sort_buffer_size.
Another problem is that ftkey_nr has no other meaning
than serial number of fulltext index starting with 1.
We can't say if this key using built-in or external
parser basing on it's value. In other words we always
entered if-branch for external parser. At this point,
the only way to check if we use default parser is to
compare keyinfo::parser with &ft_default_parser.
storage/myisam/sort.c:
Get rid of MIN_SORT_MEMORY, use MIN_SORT_BUFFER instead
(defined in myisamdef.h, has the same value and purpose).
Invalid memory read if HANDLER ... READ NEXT is executed
after failed (e.g. empty table) HANDLER ... READ FIRST.
The problem was that we attempted to perform READ NEXT,
whereas there is no pivot available from failed READ FIRST.
With this fix READ NEXT after failed READ FIRST equals
to READ FIRST.
This bug affects MyISAM tables only.
mysql-test/r/gis-rtree.result:
Restore a test case for BUG51357.
mysql-test/r/handler_myisam.result:
A test case for BUG#51877.
mysql-test/t/gis-rtree.test:
Restore a test case for BUG51357.
mysql-test/t/handler_myisam.test:
A test case for BUG#51877.
storage/myisam/mi_rnext.c:
"search first" failed. This means we have no pivot for
"search next", or in other words MI_INFO::lastkey is
likely uninitialized.
Normally SQL layer would never request "search next" if
"search first" failed. But HANDLER may do anything.
As mi_rnext() without preceeding mi_rkey()/mi_rfirst()
equals to mi_rfirst(), we must restore original state
as if failing mi_rfirst() was not called.
When MyISAM writes newly created index page it may be
initialized partially. In other words some bytes of
sensible data and uninitialized tail of the page may
go into index file.
Under certain rare circumstances these hunks of memory
may contain data that would be otherwise inaccessible
to user, like passwords or data from other tables.
Fixed by initializing memory for temporary MyISAM key
buffer to '\0'.
No test case for this fix as it is heavily covered by
existing tests.
storage/myisam/mi_open.c:
When creating new MI_INFO object, initialize MI_INFO::buff.
This is done to ensure that we never write uninitialized
memory hunks to index file.
storage/myisam/mi_page.c:
No need to silence memory error detector anymore,
page buffer is always initialized.
storage/myisam/mi_write.c:
Fixed invalid memory read of 2 bytes. new_right_length
is length of data on a page, including first 2 bytes
that store this length itself. pos + k_length is pure
data excluding these 2 bytes containing length.
storage/myisam/rt_index.c:
To avoid uninitialized data write, create new page
on info->buff, instead of locally allocated buffer.
Note: second key block on info->buff is used here,
because first block is used by called functions.
storage/myisam/rt_split.c:
To avoid uninitialized data write, create new page
on info->buff, instead of locally allocated buffer.
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/Makefile.am
Text conflict in mysql-test/collections/default.daily
Text conflict in mysql-test/r/mysqlbinlog_row_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_typeconv_innodb.result
Text conflict in mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test
Text conflict in mysql-test/suite/rpl/t/rpl_row_create_table.test
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in mysql-test/suite/rpl/t/rpl_typeconv_innodb.test
Text conflict in mysys/charset.c
Text conflict in sql/field.cc
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_func.cc
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/rpl_utility.cc
Text conflict in sql/rpl_utility.h
Text conflict in sql/set_var.cc
Text conflict in sql/share/Makefile.am
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_table.cc
Text conflict in storage/example/ha_example.h
Text conflict in storage/federated/ha_federated.cc
Text conflict in storage/myisammrg/ha_myisammrg.cc
Text conflict in storage/myisammrg/myrg_open.c
myisam tables
Queries following TRUNCATE of partitioned MyISAM table
may crash server if myisam_use_mmap is true.
Internally this is MyISAM bug, but limited to partitioned
tables, because MyISAM doesn't use ::delete_all_rows()
method for TRUNCATE, but goes via table recreate instead.
MyISAM didn't properly fall back to non-mmaped I/O after
mmap() failure. Was not repeatable on linux before, likely
because (quote from man mmap):
SUSv3 specifies that mmap() should fail if length is 0.
However, in kernels before 2.6.12, mmap() succeeded in
this case: no mapping was created and the call returned
addr. Since kernel 2.6.12, mmap() fails with the error
EINVAL for this case.
mysql-test/r/partition.result:
A test case for BUG#51868.
mysql-test/t/partition.test:
A test case for BUG#51868.
storage/myisam/mi_delete_all.c:
_mi_unmap_file() is compressed record format specific,
which is read-only. As compressed MyISAM data files are
read-only, we must never use _mi_unmap_file() in
mi_delete_all_rows().
storage/myisam/mi_dynrec.c:
Make myisam mmap code more durable to errors:
- set file_read/file_write handlers if mmap succeeded;
- reset file_read/file_write handlers on unmap.
storage/myisam/mi_extra.c:
Moved file_read/file_write handlers initialization to
mi_dynmap_file().
storage/myisam/myisamdef.h:
Added mi_munmap_file() declaration.
Conflicts:
Text conflict in mysql-test/r/partition_innodb.result
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/item_sum.h
Text conflict in sql/log_event_old.cc
Text conflict in sql/protocol.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_yacc.yy
index cardinalities=1
Parallel repair didn't poroperly update index cardinality
in certain cases.
When myisam_sort_buffer_size is not enough to store all
keys, index cardinality was updated before index was
actually written, when no index statistic is available.
mysql-test/r/myisam.result:
A test case for BUG#47444.
mysql-test/t/myisam.test:
A test case for BUG#47444.
storage/myisam/sort.c:
update_key_parts() must be called after all index
entries are written, when index statistic is available.
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/r/subselect.result
Text conflict in mysql-test/r/subselect3.result
Text conflict in mysql-test/r/type_datetime.result
Text conflict in sql/share/Makefile.am
Spatial indexes were not checking for out-of-record condition in
the handler next command when the previous command didn't found
rows.
Fixed by making the rtree index to check for end of rows condition
before re-using the key from the previous search.
Fixed another crash if the tree has changed since the last search.
Added a test case for the other error.
insert...select
Queries following bulk insert into an empty MyISAM table
may break it. This was pure MyISAM problem.
When bulk insert into an empty table is complete, MyISAM
may want to enable indexes via repair by sort. If repair
by sort fails (e.g. insufficient buffer), MyISAM failover
to repair with key cache, requesting repair of data file.
Repair of data file performs data file substitution. This
means that current table instance will point to new data
file. Other cached table instances are still pointing to
an old, deleted data file.
This is fixed by not requesting repair of data file
during enable indexes.
Explicit REPAIR is not affected, since it flushes all
table instances.
mysql-test/r/myisam.result:
A test case for BUG#51307.
mysql-test/t/myisam.test:
A test case for BUG#51307.
storage/myisam/ha_myisam.cc:
When enabling indexes do not attempt to repair data file.
This change fixes test failure for partition_repair_myisam
and partition_recover_myisam.
The problem was that:
- the file std_data/corrupt_crash.MYI
contains charsetnr=63 (binary) for index segments.
- the new .frm file contains charsetnr=8 (latin1) for
the same segments.
As a result REPAIR refused to repair frm+MYI files
with different segment definition.
This fix restores the old behavior for frm and MYI files:
they now store charsetnr=63 for numeric/datetime data types.
Fix Bug#50555 "handler commands crash server in my_hash_first()"
as a post-merge fix (the new handler tests are not passing
otherwise).
- in hash.c, don't call calc_hash if ! my_hash_inited().
- add tests and results for the test case for Bug#50555
mysys/hash.c:
Assert that the hash is initialized when it's used.
sql/set_var.cc:
Check that the hash is initalized before using it (Bug#50555)
Bulk REPLACE or bulk INSERT ... ON DUPLICATE KEY UPDATE may
break dynamic record MyISAM table.
The problem is limited to bulk REPLACE and INSERT ... ON
DUPLICATE KEY UPDATE, because only these operations may
be done via UPDATE internally and may request write cache.
When flushing write cache, MyISAM may write remaining
cached data at wrong position. Fixed by requesting write
cache to seek to a correct position.
mysql-test/r/myisam.result:
A test case for BUG#49628.
mysql-test/t/myisam.test:
A test case for BUG#49628.
storage/myisam/mi_dynrec.c:
delete_dynamic_record() may change data file position.
IO cache must be notified as it may still have cached
data, which has to be flushed later.
