mirror of
https://github.com/MariaDB/server.git
synced 2025-01-30 02:30:06 +01:00
031f11717d
The easiest way to compile and test the server with UBSAN is to run: ./BUILD/compile-pentium64-ubsan and then run mysql-test-run. After this commit, one should be able to run this without any UBSAN warnings. There is still a few compiler warnings that should be fixed at some point, but these do not expose any real bugs. The 'special' cases where we disable, suppress or circumvent UBSAN are: - ref10 source (as here we intentionally do some shifts that UBSAN complains about. - x86 version of optimized int#korr() methods. UBSAN do not like unaligned memory access of integers. Fixed by using byte_order_generic.h when compiling with UBSAN - We use smaller thread stack with ASAN and UBSAN, which forced me to disable a few tests that prints the thread stack size. - Verifying class types does not work for shared libraries. I added suppression in mysql-test-run.pl for this case. - Added '#ifdef WITH_UBSAN' when using integer arithmetic where it is safe to have overflows (two cases, in item_func.cc). Things fixed: - Don't left shift signed values (byte_order_generic.h, mysqltest.c, item_sum.cc and many more) - Don't assign not non existing values to enum variables. - Ensure that bool and enum values are properly initialized in constructors. This was needed as UBSAN checks that these types has correct values when one copies an object. (gcalc_tools.h, ha_partition.cc, item_sum.cc, partition_element.h ...) - Ensure we do not called handler functions on unallocated objects or deleted objects. (events.cc, sql_acl.cc). - Fixed bugs in Item_sp::Item_sp() where we did not call constructor on Query_arena object. - Fixed several cast of objects to an incompatible class! (Item.cc, Item_buff.cc, item_timefunc.cc, opt_subselect.cc, sql_acl.cc, sql_select.cc ...) - Ensure we do not do integer arithmetic that causes over or underflows. This includes also ++ and -- of integers. (Item_func.cc, Item_strfunc.cc, item_timefunc.cc, sql_base.cc ...) - Added JSON_VALUE_UNITIALIZED to json_value_types and ensure that value_type is initialized to this instead of to -1, which is not a valid enum value for json_value_types. - Ensure we do not call memcpy() when second argument could be null. - Fixed that Item_func_str::make_empty_result() creates an empty string instead of a null string (safer as it ensures we do not do arithmetic on null strings). Other things: - Changed struct st_position to an OBJECT and added an initialization function to it to ensure that we do not copy or use uninitialized members. The change to a class was also motived that we used "struct st_position" and POSITION randomly trough the code which was confusing. - Notably big rewrite in sql_acl.cc to avoid using deleted objects. - Changed in sql_partition to use '^' instead of '-'. This is safe as the operator is either 0 or 0x8000000000000000ULL. - Added check for select_nr < INT_MAX in JOIN::build_explain() to avoid bug when get_select() could return NULL. - Reordered elements in POSITION for better alignment. - Changed sql_test.cc::print_plan() to use pointers instead of objects. - Fixed bug in find_set() where could could execute '1 << -1'. - Added variable have_sanitizer, used by mtr. (This variable was before only in 10.5 and up). It can now have one of two values: ASAN or UBSAN. - Moved ~Archive_share() from ha_archive.cc to ha_archive.h and marked it virtual. This was an effort to get UBSAN to work with loaded storage engines. I kept the change as the new place is better. - Added in CONNECT engine COLBLK::SetName(), to get around a wrong cast in tabutil.cpp. - Added HAVE_REPLICATION around usage of rgi_slave, to get embedded server to compile with UBSAN. (Patch from Marko). - Added #ifdef for powerpc64 to avoid a bug in old gcc versions related to integer arithmetic. Changes that should not be needed but had to be done to suppress warnings from UBSAN: - Added static_cast<<uint16_t>> around shift to get rid of a LOT of compiler warnings when using UBSAN. - Had to change some '/' of 2 base integers to shift to get rid of some compile time warnings. Reviewed by: - Json changes: Alexey Botchkov - Charset changes in ctype-uca.c: Alexander Barkov - InnoDB changes & Embedded server: Marko Mäkelä - sql_acl.cc changes: Vicențiu Ciorbaru - build_explain() changes: Sergey Petrunia |
||
---|---|---|
.. | ||
bchange.c | ||
bmove_upp.c | ||
CHARSET_INFO.txt | ||
CMakeLists.txt | ||
conf_to_src.c | ||
ctype-big5.c | ||
ctype-bin.c | ||
ctype-cp932.c | ||
ctype-czech.c | ||
ctype-euc_kr.c | ||
ctype-eucjpms.c | ||
ctype-extra.c | ||
ctype-gb2312.c | ||
ctype-gbk.c | ||
ctype-latin1.c | ||
ctype-mb.c | ||
ctype-mb.ic | ||
ctype-simple.c | ||
ctype-sjis.c | ||
ctype-tis620.c | ||
ctype-uca.c | ||
ctype-uca.ic | ||
ctype-ucs2.c | ||
ctype-ucs2.h | ||
ctype-ujis.c | ||
ctype-unidata.h | ||
ctype-utf8.c | ||
ctype-utf8.h | ||
ctype-utf16.h | ||
ctype-utf32.h | ||
ctype-win1250ch.c | ||
ctype.c | ||
decimal.c | ||
do_ctype.c | ||
dtoa.c | ||
dump_map.c | ||
int2str.c | ||
is_prefix.c | ||
json_lib.c | ||
latin2.def | ||
llstr.c | ||
longlong2str.c | ||
my_strchr.c | ||
my_strtoll10.c | ||
my_vsnprintf.c | ||
README | ||
str2int.c | ||
strappend.c | ||
strcend.c | ||
strcoll.ic | ||
strcont.c | ||
strend.c | ||
strfill.c | ||
string.doc | ||
strings_def.h | ||
strmake.c | ||
strmov.c | ||
strmov_overlapp.c | ||
strnlen.c | ||
strnmov.c | ||
strxmov.c | ||
strxnmov.c | ||
t_ctype.h | ||
uca-dump.c | ||
uctypedump.c | ||
utr11-dump.c | ||
xml.c |
File : README Author : Richard A. O'Keefe. Updated: 30 April 1984 Purpose: Explain the new strings package. The UNIX string libraries (described in the string(3) manual page) differ from UNIX to UNIX (e.g. strtok is not in V7 or 4.1bsd). Worse, the sources are not in the public domain, so that if there is a string routine which is nearly what you want but not quite you can't take a copy and modify it. And of course C programmers on non-UNIX systems are at the mercy of their supplier. This package was designed to let me do reasonable things with C's strings whatever UNIX (V7, PaNiX, UX63, 4.1bsd) I happen to be using. Everything in the System III manual is here and does just what the S3 manual says it does. There are also lots of new goodies. I'm sorry about the names, but the routines do have to work on asphyxiated-at- birth systems which truncate identifiers. The convention is that a routine is called str [n] [c] <operation> If there is an "n", it means that the function takes an (int) "length" argument, which bounds the number of characters to be moved or looked at. If the function has a "set" argument, a "c" in the name indicates that the complement of the set is used. Functions or variables whose names start with _ are support routines which aren't really meant for general use. I don't know what the "p" is doing in "strpbrk", but it is there in the S3 manual so it's here too. "istrtok" does not follow this rule, but with 7 letters what can you do? I have included new versions of atoi(3) and atol(3) as well. They use a new primitive str2int, which takes a pair of bounds and a radix, and does much more thorough checking than the normal atoi and atol do. The result returned by atoi & atol is valid if and only if errno == 0. There is also an output conversion routine int2str, with itoa and ltoa as interface macros. Only after writing int2str did I notice that the str2int routine has no provision for unsigned numbers. On reflection, I don't greatly care. I'm afraid that int2str may depend on your "C" compiler in unexpected ways. Do check the code with -S. Several of these routines have "asm" inclusions conditional on the VaxAsm option. These insertions can make the routines which have them quite a bit faster, but there is a snag. The VAX architects, for some reason best known to themselves and their therapists, decided that all "strings" were shorter than 2^16 bytes. Even when the length operands are in 32-bit registers, only 16 bits count. So the "asm" versions do not work for long strings. If you can guarantee that all your strings will be short, define VaxAsm in the makefile, but in general, and when using other machines, do not define it. To use this library, you need the "strings.a" library file and the "strings.h" and "ctypes.h" header files. The other header files are for compiling the library itself, though if you are hacking extensions you may find them useful. General users really shouldn't see them. I've defined a few macros I find useful in "strings.h"; if you have no need for "index", "rindex", "streql", and "beql", just edit them out. On the 4.1bsd system I am using declaring all these functions 'extern' does not mean that they will all be loaded; but only the ones you use. When using lesser systems you may find it necessary to break strings.h up, or you could get by with just adding "extern" declarations for the functions you want as you need them. Many of these functions have the same names as functions in the "standard C library", by design as this is a replacement/reimplementation of part of that library. So you may have to talk the loader into loading this library first. Again, I've found no problems on 4.1bsd. You may wonder at my failure to provide manual pages for this code. For the things in V7, 4.?, or SIII, you should be able to use whichever manual page came with that system, and anything I might write would be so like it as to raise suspicions of violating AT&T copyrights. In the sources you will find comments which provide far more documentation for these routines than AT&T ever provided for their strings stuff, I just don't happen to have put it in nroff -man form. Had I done so, the .3 files would have outbulked the .c files! These files are in the public domain. This includes getopt.c, which is the work of Henry Spencer, University of Toronto Zoology, who says of it "None of this software is derived from Bell software. I had no access to the source for Bell's versions at the time I wrote it. This software is hereby explicitly placed in the public domain. It may be used for any purpose on any machine by anyone." I would greatly prefer it if *my* material received no military use.