2677 Vladislav Vaintroub 2008-11-04
CMakeLists.txt files cleanup
- remove SAFEMALLOC and SAFE_MUTEX definitions that were
present in *each* CMakeLists.txt. Instead, put them into top level
CMakeLists.txt, but disable on Windows, because
a) SAFEMALLOC does not add any functionality that is not already
present in Debug C runtime ( and 2 safe malloc one on top of the other
only unnecessarily slows down the server)
b)SAFE_MUTEX does not work on Windows and have been
explicitely disabled on Windows with #undef previously. Fortunately,
ntdll does pretty good job identifying l problems with
CRITICAL_SECTIONs.
DebugBreak()s on using uninited critical section, unlocking unowned
critical section)
-Also, remove occationally used -D_DEBUG (added by compiler
anyway)
This patch provides performance improvements:
- send_fields() when character_set_results = latin1
is now about twice faster for column/table/database
names, consisting on ASCII characters.
Changes:
- Protocol doesn't use "convert" temporary buffer anymore,
and converts strings directly to "packet".
- General conversion optimization: quick conversion
of ASCII strings was added.
modified files:
include/m_ctype.h
- Adding a new flag.
- Adding a new function prototype
libmysqld/lib_sql.cc
- Adding quick conversion method for embedded library:
conversion is now done directly to result buffer,
without using a temporary buffer.
mysys/charset.c
- Mark all dynamic ucs2 character sets as non-ASCII
- Mark some dymamic 7bit and 8bit charsets as non-ASCII
(for example swe7 is not fully ASCII compatible).
sql/protocol.cc
- Adding quick method to convert a string directly
into protocol buffer, without using a temporary buffer.
sql/protocol.h
- Adding a new method prototype
sql/sql_string.cc
Optimization for conversion between two ASCII-compatible charsets:
- quickly convert ASCII strings,
switch to mc_wc->wc_mb method only when a non-ASCII character is met.
- copy four ASCII characters at once on i386
strings/conf_to_src.c
- Marking non-ASCII character sets with a flag.
strings/ctype-extra.c
- Regenerating ctype-extra.c by running "conf_to_src".
strings/ctype-uca.c
- Marking UCS2 character set as non-ASCII.
strings/ctype-ucs2.c
- Marking UCS2 character set as non-ASCII.
strings/ctype.c
- A new function to detect if a 7bit or 8bit character set
is ascii compatible.
with gcc 4.3.2
This patch fixes a number of GCC warnings about variables used
before initialized. A new macro UNINIT_VAR() is introduced for
use in the variable declaration, and LINT_INIT() usage will be
gradually deprecated. (A workaround is used for g++, pending a
patch for a g++ bug.)
GCC warnings for unused results (attribute warn_unused_result)
for a number of system calls (present at least in later
Ubuntus, where the usual void cast trick doesn't work) are
also fixed.
Problem:
Crash happened with a user-defined utf8 collation,
on attempt to insert a value longer than the column
to store.
Reason:
The "ctype" member was not initialized (NULL) when
allocating a user-defined utf8 collation, so an attempt
to call my_ctype(cs, *str) to check if we loose any important
data when truncating the value made the server crash.
Fix:
Initializing tge "ctype" member to a proper value.
mysql-test/r/ctype_ldml.result
Adding tests
mysql-test/t/ctype_ldml.test
Adding tests
strings/ctype-uca.c
Adding initialization of "ctype" member.
modified:
mysql-test/r/ctype_ldml.result
mysql-test/t/ctype_ldml.test
strings/ctype-uca.c
on cp932 and sjis environment.
Problem: case conversion erroneously changes the second bytes
of multi-byte sequences because single-byte functions were
called in a mistake.
Fix: call multi-byte aware functions instead.
The reference manual has instructions for adding new character
sets, and refers to the string/CHARSET_INFO.txt file. This file
is currently not present in the distribution.
Modify the build to include this file in the distribution.
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
- Remove bothersome warning messages. This change focuses on the warnings
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
The MONTHNAME/DAYNAME functions
returns binary string, so the LOWER/UPPER functions
are not effective on the result of MONTHNAME/DAYNAME call.
Character set of the MONTHNAME/DAYNAME function
result has been changed to connection character set.
Problem:
XML syntax parser allowed to use quoted strings as attribute names,
and tried to put them into parser state stack instead of identifiers.
After that parser failed, if quoted string contained some slash characters.
Fix:
- Disallowing quoted strings in regular tags.
- Allowing quoted string in DOCTYPE declararion, but
don't push it into parse state stack (just skip it).
When the fractional part in a multiplication of DECIMALs
overflowed, we truncated the first operand rather than the
longest. Now truncating least significant places instead
for more precise multiplications.
(Queuing at demand of Trudy/Davi.)
Grouping or ordering of long values in not indexed BLOB/TEXT columns
with GBK or BIG5 charsets crashes the server.
MySQL server uses sorting (the filesort procedure) in the temporary
table to evaluate the GROUP BY clause in case of lack of suitable index.
That procedure takes into account only first @max_sort_length bytes
(system variable, usually 1024) of TEXT/BLOB sorting key string.
The my_strnxfrm_gbk and my_strnxfrm_big5 fill temporary keys
with data of whole blob length instead of @max_sort_length bytes
length. That buffer overrun has been fixed.
Problem: some collation handlers called incorrect version
of my_like_range_xxx(), which led to wrong min_str and max_str,
so like range optimizer threw away good records.
Fix: changing the wrong handlers to call proper version of
my_like_range_xxx().
The ROUND(X, D) function would change the Item::decimals field during
execution to achieve the effect of a dynamic number of decimal digits.
This caused a series of bugs:
Bug #30617:Round() function not working under some circumstances in InnoDB
Bug #33402:ROUND with decimal and non-constant cannot round to 0 decimal places
Bug #30889:filesort and order by with float/numeric crashes server
Fixed by never changing the number of shown digits for DECIMAL when
used with a nonconstant number of decimal digits.
and vice versa"
Initial import of the dtoa.c code and custom wrappers around it to allow
its usage from the server code.
Conversion of FLOAT/DOUBLE values to DECIMAL ones or strings and vice
versa has been significantly reworked. As the new algoritms are more
precise than the older ones, results of such conversions may not always
match those obtained from older server versions. This in turn may break
compatibility for some applications.
This patch also fixes the following bugs:
- bug #12860 "Difference in zero padding of exponent between Unix and
Windows"
- bug #21497 "DOUBLE truncated to unusable value"
- bug #26788 "mysqld (debug) aborts when inserting specific numbers into
char fields"
- bug #24541 "Data truncated..." on decimal type columns without any
good reason"
Default values of variables were not subject to upper/lower bounds
and step, while setting variables was. Bounds and step are also
applied to defaults now; defaults are corrected quietly, values
given by the user are corrected, and a correction-warning is thrown
as needed. Lastly, very large values could wrap around, starting
from 0 again. They are bounded at the maximum value for the
respective data-type now if no lower maximum is specified in the
variable's definition.
strmake() calls are easy to get wrong. Add checks in extra
debug mode to identify possible exploits.
Remove some dead code.
Remove some off-by-one errors identified with new checks.
An integer overflow in number->string conversion caused completely
wrong output of the number LONGLONG_MIN with gcc 4.2.1.
Fixed by eliminating the overflow, using only operations that are
well-defined in ANSI C.
Problem:
my_strntoull10rnd_8bit() handled incorrectly cases when the input
string contains a decimal point and is long enough to overrun the
'unsigned long long' type. The position of the decimal point was not
taken into account which resulted in miscalculated numbers and
truncation to appropriate SQL data type limits.
Solution:
Fix my_strntoull10rnd_8bit() to take the position of a decimal point
into account in such cases.
when a divisor is less than 1 and its fractional part is very long.
For example:
1 % .123456789123456789123456789123456789123456789123456789123456789123456789123456789;
Stack buffer overflow has been fixed in the do_div_mod function.
and for bug #31070: crash during conversion of charsets
Problem: passing a 0 byte length string to some my_mb_wc_XXX()
functions leads to server crash due to improper argument check.
Fix: properly check arguments passed to my_mb_wc_XXX() functions.
Problem: some valid euc-kr characters were rejected because
condition checking multi-byte tail didn't allow
multi-byte characters having the second byte in the ranges
[0x41..0x5A] and [0x61..0x7A].
Fix: allow these byte ranges for mb tails
Faster thr_alarm()
Added 'Opened_files' status variable to track calls to my_open()
Don't give warnings when running mysql_install_db
Added option --source-install to mysql_install_db
I had to do the following renames() as used polymorphism didn't work with Forte compiler on 64 bit systems
index_read() -> index_read_map()
index_read_idx() -> index_read_idx_map()
index_read_last() -> index_read_last_map()
Two character mappings were way off (backtick and tilde were "E"
and "Y"!), and three others were slightly rotated. The first
would cause collisions, and the latter was probably benign.
Now, assign the character mappings exactly to their normal values.
Additional changes for bug#29903
- Changed to do embedded build part as normal build, when
WITH_EMBEDDED_SERVER is set.
- Allow both normal and debug build with embedded.
- Build static embedded library by pointing out all source and compile
it all, i.e. not building libraries from libraries, not portable.
- Let embedded use generated files from the "sql" directory, added
dependencies to make sure built before embedded.
- Mark library "dbug" in TARGET_LINK_LIBRARIES() with "debug", so only
linked in when debug target is used.
- Removed change of target name with "mysqld${MYSQLD_EXE_SUFFIX}", as
others can't depend on it, not defined at configure time. Instead
set the output file name.
- Created work around for bug in CMake 2.4.6 and output names, to
set the "mysqld<suffix>.pdb" name to the same base name.
- Set the correct manifest "name" (patch by iggy)
(Regression, caused by a patch for the bug 22646).
Problem: when result type of date_format() was changed from
binary string to character string, mixing date_format()
with a ascii column in CONCAT() stopped to work.
Fix:
- adding "repertoire" flag into DTCollation class,
to mark items which can return only pure ASCII strings.
- allow character set conversion from pure ASCII to other character sets.
They had been introduced in 5.1 and were only later backported to 5.0;
as a consequence, the files in the 5.1 tree do not depend on the 5.0 ones,
and changes in 5.0 do not propagate into the 5.1 files.
To fix this, the (previous) files in 5.1 now are deleted ("bk rm"),
and the previously deleted files depending on 5.0 are now moved to the
respective source directories ("bk mv").
The current 5.1 contents is restored in these files.
If you need the previous history of the 5.1 files ("bk revtool"),
access those in "BitKeeper/deleted".
Contrary to the original plan, I did not introduce the name
"CMakeLists.historic" - mostly in order not to clutter the source tree.
This fixes bug#29982.
with the space character.
When the my_strnncollsp_simple function compares two strings and one is a prefix
of another then this function compares characters in the rest of longer key
with the space character to find whether the longer key is greater or less.
But the sort order of the collation isn't used in this comparison. This may
lead to a wrong comparison result, wrongly created index or wrong order of the
result set of a query with the ORDER BY clause.
Now the my_strnncollsp_simple function uses collation sort order to compare
the characters in the rest of longer key with the space character.
Problem: wrong comparison with trailing space.
This problem was fixed for all other character sets under terms of
bug 7788 ""Table is full" occurs during a multitable update".
ctype-cp932.c was forgotten.
Fix: applying the same fix for ctype-cp932.c.
(see ctype-sjis.c as an example of a previously correctly fixed file)
spaces.
When the my_strnncollsp_simple function compares two strings and one is a prefix
of another then this function compares characters in the rest of longer key
with the space character to find whether the longer key is greater or less.
But the sort order of the collation isn't used in this comparison. This may
lead to a wrong comparison result, wrongly created index or wrong order of the
result set of a query with the ORDER BY clause.
Now the my_strnncollsp_simple function uses collation sort order to compare
the characters in the rest of longer key with the space character.
Problem: like_range() returned wrong ranges for contractions (like 'ch' in Czech').
Fix: adding a special code to handle tricky cases:
- contraction head followed by a wild character
- full contraction
- contraction part followed by another contraction part,
but they are not a contraction together.
and is not described in the manual
- Adding missing initialization for utf8 collations
- Minor code clean-ups: renaming variables,
moving code into a new separate function.
- Adding test, to check that both ucs2 and utf8 user
defined collations work (ucs2_test_ci and utf8_test_ci)
- Adding Vietnamese collation as a complex user defined
collation example.
Problem: "SELECT INTO OUTFILE" created incorrect dumps for BLOBs,
so "LOAD DATA" later incorrectly interpreted 0x5C as the second
byte of a multi-byte sequence, instead of escape character.
Fix: adding escaping of multi-byte heads.
---
Added casts and fixed wrong type.
---
Added casts and fixed wrong type.
---
Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-5.1-marvel
into a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-5.1-marvel
---
Don't give warning that readonly variable is forced to be readonly
mysql-test-run run now fails if we have [Warning] and [ERROR] as tags in .err file
Fixed wrong reference to the mysql manual
Fixed wrong prototype that caused some tests to fail on 64 bit platforms
---
Disabled compiler warnings mainly for Win 64.
---
Added casts to remove compiler warnings on windows
Give warnings also for safe_mutex errors found by test system
Added some warnings from different machines in pushbuild
---
Merge bk-internal.mysql.com:/home/bk/mysql-5.1-marvel
into mysql.com:/home/my/mysql-5.1
---
Added escapes for double quotes and parenthesis.
---
Archive db fix plus added non-critical warnings
in ignore list.
---
Fixed previously added patch and added new ignored warning.
Problem:
HASH indexes on VARCHAR columns with binary collations did not ignore trailing spaces from strings before comparisons. This could result in duplicate records being successfully inserted into a MEMORY table with unique key constraints.
As a direct consequence of the above, internal MEMORY tables used for GROUP BY calculation in testcases for bug #27643 contained duplicate rows which resulted in duplicate key errors when converting those temporary tables to MyISAM. Additionally, that error was incorrectly converted to the 'table is full' error.
Solution:
- ignore trailing spaces in VARCHAR fields with binary collations when calculating hashes.
- return a proper error from create_myisam_from_heap() when conversion fails.
Problem: Unicode->UJIS followed incorrect conversion
rules for U+00A5 YEN SIGN and U+203E OVERLINE,
so these characters were converted to ujis 0x8E5C
and 0x8E7E accordingly.
This behaviour would be correct for a JIS-X-0201 based character set,
but this is wrong for UJIS, which is documented as x-eucjp-unicode-0.9,
and which is based on ASCII for the range U+0000..U+007F.
Fix:
removing JIS-X-0201 conversion rules, making UJIS ASCII compatible.
YEN SIGN and OVERLINE do not have corresponding UJIS characters anymore
and converted to 0x3F QUESTION MARK, throwing a warning in appropriative cases.
This patch also includes a test covering full UJIS->Unicode->UJIS mapping.
