This patch fixes two problems:
- The code inside my_strtod_int() in strings/dtoa.c could test the byte
behind the end of the string when processing the mantissa.
Rewriting the code to avoid this.
- The code in test_if_number() in sql/sql_analyse.cc called my_atof()
which is unsafe and makes the called my_strtod_int() look behind
the end of the string if the input string is not 0-terminated.
Fixing test_if_number() to use my_strtod() instead, passing the correct
end pointer.
Fixing a few problems relealed by UBSAN in type_float.test
- multiplication overflow in dtoa.c
- uninitialized Field::geom_type (and Field::srid as well)
- Wrong call-back function types used in combination with SHOW_FUNC.
Changes in the mysql_show_var_func data type definition were not
properly addressed all around the code by the following commits:
b4ff64568c18feb62fee0ee879ff8a
Adding a helper SHOW_FUNC_ENTRY() function and replacing
all mysql_show_var_func declarations using SHOW_FUNC
to SHOW_FUNC_ENTRY, to catch mysql_show_var_func in the future
at compilation time.
MDEV-22691 MSAN use-of-uninitialized-value in test maria.maria-recovery2
This caused all my_vsnprintf() using doubles to fail.
Thanks to the workaround, I was able to remove the disabling of
MSAN in dtoa().
MemorySanitizer (clang -fsanitize=memory) requires that all code
be compiled with instrumentation enabled. The C runtime library
is an exception. Failure to use instrumented libraries will cause
bogus messages about memory being uninitialized.
In WITH_MSAN builds, we must avoid calling getservbyname(),
because even though it is a standard library function, it is
not instrumented, not even in clang 10.
The following cmake options were tested:
-DCMAKE_C_FLAGS='-march=native -O2'
-DCMAKE_CXX_FLAGS='-stdlib=libc++ -march=native -O2'
-DWITH_EMBEDDED_SERVER=OFF -DWITH_UNIT_TESTS=OFF -DCMAKE_BUILD_TYPE=Debug
-DWITH_INNODB_{BZIP2,LZ4,LZMA,LZO,SNAPPY}=OFF
-DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,CONNECT,SPIDER}=NO
-DWITH_SAFEMALLOC=OFF
-DWITH_{ZLIB,SSL,PCRE}=bundled
-DHAVE_LIBAIO_H=0
-DWITH_MSAN=ON
MEM_MAKE_DEFINED(): An alias for VALGRIND_MAKE_MEM_DEFINED()
and in the future, __msan_unpoison().
For now, neither MEM_MAKE_DEFINED() nor MEM_UNDEFINED()
perform any action under MSAN. Enabling them will catch more bugs, but
will also require some more fixes or work-arounds.
Json_writer::add_double(): Work around a frequently occurring
failure in optimizer tests, related to EXPLAIN FORMAT=JSON.
dtoa(): Disable MSAN altogether. For some reason, this function
is triggering a lot of trouble, especially when invoked for
DBUG functions. The MDL default timeout is dd=86400 seconds,
and for some reason it is claimed to be uninitialized.
InnoDB: Define UNIV_DEBUG_VALGRIND also WITH_MSAN.
ut_crc32_8_hw(), ut_crc32_64_low_hw(): Use the compiler built-in
functions instead of inline assembler when building WITH_MSAN.
This will require at least -msse4.2 when building for IA-32 or AMD64.
The inline assembler would not be instrumented, and would thus cause
bogus failures.
Do not silence uncertain cases, or fix any bugs.
The only functional change should be that ha_federated::extra()
is not calling DBUG_PRINT to report an unhandled case for
HA_EXTRA_PREPARE_FOR_DROP.
Do not silence uncertain cases, or fix any bugs.
The only functional change should be that ha_federated::extra()
is not calling DBUG_PRINT to report an unhandled case for
HA_EXTRA_PREPARE_FOR_DROP.
Decimals with float, double and decimal now works the following way:
- DECIMAL_NOT_SPECIFIED is used when declaring DECIMALS without a firm number
of decimals. It's only used in asserts and my_decimal_int_part.
- FLOATING_POINT_DECIMALS (31) is used to mark that a FLOAT or DOUBLE
was defined without decimals. This is regarded as a floating point value.
- Max decimals allowed for FLOAT and DOUBLE is FLOATING_POINT_DECIMALS-1
- Clients assumes that float and double with decimals >= NOT_FIXED_DEC are
floating point values (no decimals)
- In the .frm decimals=FLOATING_POINT_DECIMALS are used to define
floating point for float and double (31, like before)
To ensure compatibility with old clients we do:
- When storing float and double, we change NOT_FIXED_DEC to
FLOATING_POINT_DECIMALS.
- When creating fields from .frm we change for float and double
FLOATING_POINT_DEC to NOT_FIXED_DEC
- When sending definition for a float/decimal field without decimals
to the client as part of a result set we convert NOT_FIXED_DEC to
FLOATING_POINT_DECIMALS.
- variance() and std() has changed to limit the decimals to
FLOATING_POINT_DECIMALS -1 to not get the double converted floating point.
(This was to preserve compatiblity)
- FLOAT and DOUBLE still have 30 as max number of decimals.
Bugs fixed:
variance() printed more decimals than we support for double values.
New behaviour:
- Strings now have 38 decimals instead of 30 when converted to decimal
- CREATE ... SELECT with a decimal with > 30 decimals will create a column
with a smaller range than before as we are trying to preserve the number of
decimals.
Other changes
- We are now using the obsolete bit FIELDFLAG_LEFT_FULLSCREEN to specify
decimals > 31
- NOT_FIXED_DEC is now declared in one place
- For clients, NOT_FIXED_DEC is always 31 (to ensure compatibility).
On the server NOT_FIXED_DEC is DECIMAL_NOT_SPECIFIED (39)
- AUTO_SEC_PART_DIGITS is taken from DECIMAL_NOT_SPECIFIED
- DOUBLE conversion functions are now using DECIMAL_NOT_SPECIFIED instead of
NOT_FIXED_DEC
mysql-test/suite/innodb/t/group_commit_crash.test:
remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
my_vsnprintf() is ok here, in 5.5
Bug#12985021 SIMPLE QUERY WITH DECIMAL NUMBERS TAKE AN
When parsing the fractional part of a string which
is to be converted to double, we can stop after a few digits:
the extra digits will not contribute to the actual result anyways.
mysql-test/r/func_str.result:
New tests.
mysql-test/t/func_str.test:
New tests.
strings/dtoa.c:
The problem was s2b() multiplying and adding hundreds-of-thousands
of ever smaller fractions.
Fixed memory leak printing when doing 'mysqld --version', 'mysqld --debug --help' and 'mysqld --debug --help --verbose'
mysys/my_init.c:
Moved checking if we should call DBUG_END() before my_thread_end() as otherwise we will not free DBUG variables and files.
mysys/thr_lock.c:
Fixed compiler warning
sql/mysqld.cc:
Fixed memory leaks when using mysqld --help and mysqld --version
Added --debug as an option that works for all builds. For non debug builds we now get a warning.
strings/dtoa.c:
Fixed valgrind warning (c could contain data outside of the given string)
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
Extra fix: 'if (p5 < p5_a + P5A_MAX)' is not portable.
p5 starts out pointing to a static array, then may point
to a buffer on the stack, then may point to malloc()ed memory.
mysql-test/r/func_str.result:
New test cases.
mysql-test/t/func_str.test:
New test cases.
strings/dtoa.c:
Increasing the buffer size slightly made some queries pass without leaks.
Adding Bfree(p51, alloc) fixed the remaining leaks.
- Ported relevant changes from the upstream version to not
break strict-aliasing rules and to fix compiler warnings and
and infinite loops caused by that issue.
- Fixed compilation with Honor_FLT_ROUNDS defined.
- Fixed an unused variable warning.
The failing assertion was written with the assumption that a NULL
string can never be passed to my_strtod(). However, an empty string
may be passed under some circumstances by passing str == NULL and
*end == NULL.
Fixed the assertion to take the above case into account.
mysql-test/r/func_misc.result:
Added a test case for bug #52165.
mysql-test/t/func_misc.test:
Added a test case for bug #52165.
strings/dtoa.c:
Fixed the assertion in my_strtod() to take the case of 'str == NULL
&& *end == NULL' into account.
to string conversions and vice versa"
Initial import of the dtoa.c code and custom wrappers around it
to allow its usage from the server code.
Conversion of FLOAT/DOUBLE values to DECIMAL ones or strings
and vice versa has been significantly reworked. As the new
algoritms are more precise than the older ones, results of such
conversions may not always match those obtained from older
server versions. This in turn may break compatibility for some
applications.
This patch also fixes the following bugs:
- bug #12860 "Difference in zero padding of exponent between
Unix and Windows"
- bug #21497 "DOUBLE truncated to unusable value"
- bug #26788 "mysqld (debug) aborts when inserting specific
numbers into char fields"
- bug #24541 "Data truncated..." on decimal type columns
without any good reason"
and vice versa"
Initial import of the dtoa.c code and custom wrappers around it to allow
its usage from the server code.
Conversion of FLOAT/DOUBLE values to DECIMAL ones or strings and vice
versa has been significantly reworked. As the new algoritms are more
precise than the older ones, results of such conversions may not always
match those obtained from older server versions. This in turn may break
compatibility for some applications.
This patch also fixes the following bugs:
- bug #12860 "Difference in zero padding of exponent between Unix and
Windows"
- bug #21497 "DOUBLE truncated to unusable value"
- bug #26788 "mysqld (debug) aborts when inserting specific numbers into
char fields"
- bug #24541 "Data truncated..." on decimal type columns without any
good reason"
BitKeeper/deleted/.del-strtod.c:
Rename: strings/strtod.c -> BitKeeper/deleted/.del-strtod.c
client/Makefile.am:
Added dtoa.c
client/sql_string.cc:
Replaced sprintf() with a new wrapper around dtoa()
include/m_string.h:
Added declarations for the new double to/from string conversion
functions.
libmysql/Makefile.shared:
Removed strtod.c, added dtoa.c
libmysql/libmysql.c:
Replaced sprintf() with my_gcvt(). We must use the same conversion
method in both server and client lib.
mysql-test/r/archive_gis.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/cast.result:
Fixed tests to take the formatting changes into account.
mysql-test/r/func_group.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/func_math.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/func_str.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/gis.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/innodb_gis.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/insert.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/mysqldump.result:
No need for result substitution since we do not rely on system behavior
anymore.
mysql-test/r/ndb_gis.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/parser.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/ps_2myisam.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/ps_3innodb.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/ps_4heap.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/ps_5merge.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/ps_7ndb.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/select.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/r/sp.result:
Fixed tests to take additional precision from dtoa() into account.
Decimal 2.7182818284590452354 is now converted to the double value of
2.718281828459045, not 2.718281828459. Thus we get additional precision
from the subsequent calculations.
mysql-test/r/strict.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/type_decimal.result:
Added a test case for bug #24541.
mysql-test/r/type_float.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
Added test cases for bugs #12860, #21497 and #26788.
mysql-test/r/type_newdecimal.result:
Fixed tests to take additional precision from dtoa() and formatting
changes into account.
mysql-test/r/variables.result:
Fixed tests to take the formatting changes into account.
mysql-test/r/view.result:
Fixed tests to take additional precision from dtoa() into account.
mysql-test/t/cast.test:
No need to replace the results for Windows anymore.
mysql-test/t/insert.test:
Added test cases from bug #26788.
mysql-test/t/mysqldump.test:
Fixed tests to take additional precision from dtoa() into account.
No need for results substitution because we do not rely on system
behavior anymore.
mysql-test/t/type_decimal.test:
Added a test case for bug #24541.
mysql-test/t/type_float.test:
Added test cases for bugs #12860, #21497 and #26788.
mysql-test/t/type_newdecimal.test:
Removed replace_result which is pointless with our own floating point
conversion library.
Added a reference for WL#3977.
sql/field.cc:
Replaced the field.cc's own constant for the maximum double->string
conversion buffer with the one defined in m_string.h
Replaced double->string conversion code with the new wrappers around
dtoa().
Fixed a bug in Field_real::truncate() which led to incorrect results
when +-Infinity was passed as an input number.
Fixed Field_blob::store(double) to use NOT_FIXED_DEC instead of 2 as
precision, so that my_gcvt() is now used for conversion.
sql/field.h:
Moved NOT_FIXED_DEC to m_string.h because we now use this constant in
floating point <-> string conversion routines.
sql/item.cc:
If a result of a numeric functions is stored in a string field, follow
the same rules as in Field_str::store(double), i.e. take the field width
into account.
sql/log_event.cc:
Replaced sprintf() with my_gcvt().
sql/mysql_priv.h:
Moved log_10[] from strtod.c to mysqld.c, because we don't need it in
libmystrings anymore.
sql/mysqld.cc:
Moved log_10[] from strtod.c to mysqld.c, because we don't need it in
libmystrings anymore.
sql/sql_show.cc:
Replaced sprintf() with my_fcvt().
sql/sql_string.cc:
Replace the double->string conversion code with the new wrappers around
dtoa().
sql/unireg.h:
Moved FLOATING_POINT_BUFFER from unireg.h to m_string.h so it can be
used in libmystrings.
strings/Makefile.am:
Removed strtod.c, added dtoa.c
strings/decimal.c:
Replaced sprintf() with my_gcvt().
Changed double2decimal() to print the digits to a string buffer, then
use my_strtod().
strings/dtoa.c:
Initial import of the dtoa code and custom wrappers around it to allow
its usage from the server code.
