mariadb/strings
Marko Mäkelä 94d0bb4dbe MDEV-20377: Make WITH_MSAN more usable
MemorySanitizer (clang -fsanitize=memory) requires that all code
be compiled with instrumentation enabled. The C runtime library
is an exception. Failure to use instrumented libraries will cause
bogus messages about memory being uninitialized.

In WITH_MSAN builds, we must avoid calling getservbyname(),
because even though it is a standard library function, it is
not instrumented, not even in clang 10.

The following cmake options were tested:

-DCMAKE_C_FLAGS='-march=native -O2'
-DCMAKE_CXX_FLAGS='-stdlib=libc++ -march=native -O2'
-DWITH_EMBEDDED_SERVER=OFF -DWITH_UNIT_TESTS=OFF -DCMAKE_BUILD_TYPE=Debug
-DWITH_INNODB_{BZIP2,LZ4,LZMA,LZO,SNAPPY}=OFF
-DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,CONNECT,SPIDER}=NO
-DWITH_SAFEMALLOC=OFF
-DWITH_{ZLIB,SSL,PCRE}=bundled
-DHAVE_LIBAIO_H=0
-DWITH_MSAN=ON

MEM_MAKE_DEFINED(): An alias for VALGRIND_MAKE_MEM_DEFINED()
and in the future, __msan_unpoison().

For now, neither MEM_MAKE_DEFINED() nor MEM_UNDEFINED()
perform any action under MSAN. Enabling them will catch more bugs, but
will also require some more fixes or work-arounds.

Json_writer::add_double(): Work around a frequently occurring
failure in optimizer tests, related to EXPLAIN FORMAT=JSON.

dtoa(): Disable MSAN altogether. For some reason, this function
is triggering a lot of trouble, especially when invoked for
DBUG functions. The MDL default timeout is dd=86400 seconds,
and for some reason it is claimed to be uninitialized.

InnoDB: Define UNIV_DEBUG_VALGRIND also WITH_MSAN.

ut_crc32_8_hw(), ut_crc32_64_low_hw(): Use the compiler built-in
functions instead of inline assembler when building WITH_MSAN.
This will require at least -msse4.2 when building for IA-32 or AMD64.
The inline assembler would not be instrumented, and would thus cause
bogus failures.
2020-03-28 21:36:30 +02:00
..
bchange.c
bmove_upp.c
CHARSET_INFO.txt
CMakeLists.txt
conf_to_src.c
ctype-big5.c
ctype-bin.c Fixed compiler warnings from gcc 7.4.1 2020-01-29 23:23:55 +02:00
ctype-cp932.c
ctype-czech.c
ctype-euc_kr.c
ctype-eucjpms.c
ctype-extra.c
ctype-gb2312.c
ctype-gbk.c
ctype-latin1.c
ctype-mb.c MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
ctype-mb.ic
ctype-simple.c Merge 10.4 into 10.5 2020-02-07 08:51:20 +02:00
ctype-sjis.c
ctype-tis620.c perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00
ctype-uca.c MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
ctype-uca.ic MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
ctype-ucs2.c MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
ctype-ucs2.h
ctype-ujis.c
ctype-unidata.h
ctype-utf8.c MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
ctype-utf8.h
ctype-utf16.h
ctype-utf32.h
ctype-win1250ch.c
ctype.c MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
decimal.c
do_ctype.c
dtoa.c MDEV-20377: Make WITH_MSAN more usable 2020-03-28 21:36:30 +02:00
dump_map.c
int2str.c
is_prefix.c
json_lib.c Merge 10.4 into 10.5 2020-03-17 07:58:41 +02:00
latin2.def
llstr.c
longlong2str.c
my_strchr.c MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
my_strtoll10.c
my_vsnprintf.c perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00
README
str2int.c
strappend.c
strcend.c
strcoll.ic Merge 10.4 into 10.5 2020-02-07 08:51:20 +02:00
strcont.c
strend.c
strfill.c
string.doc
strings_def.h
strmake.c
strmov.c
strmov_overlapp.c
strnlen.c
strnmov.c
strxmov.c
strxnmov.c
t_ctype.h
uca-dump.c
uctypedump.c
utr11-dump.c
xml.c perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00

File   : README
Author : Richard A. O'Keefe.
Updated: 30 April 1984
Purpose: Explain the new strings package.

    The UNIX string libraries (described in the string(3) manual page)
differ from UNIX to UNIX (e.g. strtok is not in V7 or 4.1bsd).  Worse,
the sources are not in the public domain, so that if there is a string
routine which is nearly what you want but not quite you can't  take  a
copy  and  modify it.  And of course C programmers on non-UNIX systems
are at the mercy of their supplier.

    This package was designed to let me do reasonable things with  C's
strings  whatever UNIX (V7, PaNiX, UX63, 4.1bsd) I happen to be using.
Everything in the System III manual is here and does just what the  S3
manual  says  it does.  There are also lots of new goodies.  I'm sorry
about the names, but the routines do have to work  on  asphyxiated-at-
birth  systems  which  truncate identifiers.  The convention is that a
routine is called
 str [n] [c] <operation>
If there is an "n", it means that the function takes an (int) "length"
argument, which bounds the number of characters to be moved or  looked
at.  If the function has a "set" argument, a "c" in the name indicates
that  the complement of the set is used.  Functions or variables whose
names start with _ are support routines which aren't really meant  for
general  use.  I don't know what the "p" is doing in "strpbrk", but it
is there in the S3 manual so it's here too.  "istrtok" does not follow
this rule, but with 7 letters what can you do?

    I have included new versions of atoi(3) and atol(3) as well.  They
use a new primitive str2int, which takes a pair of bounds and a radix,
and does much more thorough checking than the normal atoi and atol do.
The result returned by atoi & atol is valid if and only if errno == 0.
There is also an output conversion routine int2str, with itoa and ltoa
as interface macros.  Only after writing int2str did I notice that the
str2int routine has no provision for unsigned numbers.  On reflection,
I don't greatly care.   I'm afraid that int2str may depend on your "C"
compiler in unexpected ways.  Do check the code with -S.

    Several of these routines have "asm" inclusions conditional on the
VaxAsm option.  These insertions can make the routines which have them
quite a bit faster, but there is a snag.  The VAX architects, for some
reason best known to themselves and their therapists, decided that all
"strings" were shorter than 2^16 bytes.  Even when the length operands
are in 32-bit registers, only 16 bits count.  So the "asm" versions do
not work for long strings.  If you can guarantee that all your strings
will be short, define VaxAsm in the makefile, but in general, and when
using other machines, do not define it.

    To use this library, you need the "strings.a" library file and the
"strings.h" and "ctypes.h" header files.  The other header  files  are
for compiling the library itself, though if you are hacking extensions
you  may  find  them useful.  General users really shouldn't see them.
I've defined a few macros I find useful in "strings.h"; if you have no
need for "index", "rindex", "streql", and "beql", just edit them  out.
On the 4.1bsd system I am using declaring all these functions 'extern'
does not mean that they will all be loaded; but only the ones you use.
When using lesser systems you may find it necessary to break strings.h
up, or you could get by with just adding "extern" declarations for the
functions you want as you need them.  Many of these functions have the
same names as functions in the "standard C library", by design as this
is a replacement/reimplementation of part of that library.  So you may
have to talk the loader into loading this library first.   Again, I've
found no problems on 4.1bsd.

    You may wonder at my failure to provide manual pages for this code.
For the things in V7, 4.?, or SIII, you should be able to use whichever
manual page came with that system,  and anything I might write would be
so like it as to raise suspicions of violating AT&T copyrights.  In the
sources you will find comments which provide far more documentation for
these routines than AT&T ever provided for their strings stuff,  I just
don't happen to have put it in nroff -man form.   Had I done so, the .3
files would have outbulked the .c files!

    These files are in the public domain.  This includes getopt.c, which
is the work of Henry Spencer, University of Toronto Zoology, who says of
it "None of this software is derived from Bell software. I had no access
to the source for Bell's versions at the time I wrote it.  This software
is hereby explicitly placed in the public domain.  It may  be  used  for
any purpose on any machine by anyone." I would greatly prefer it if *my*
material received no military use.