In fact it was error in decimal library (incorrect processing of buffer overflow) invisible from other server parts because of buffer allocation and precision tests.
COLLATIONS ARE USED.
ISSUE :
-------
Code points of HALF WIDTH KATAKANA in SJIS/CP932 range from
A1 to DF. In function my_wildcmp_mb_bin_impl while comparing
such single byte code points, there is a code which compares
signed character with unsigned character. Because of this,
comparisons of two same code points representing a HALF
WIDTH KATAKANA character always fails.
Solution:
---------
A code point of HALF WIDTH KATAKANA at-least need 8 bits.
Promoting the variable from uchar to int will fix the issue.
mysql-test/t/ctype_cp932.test:
Tests which have conditions
LIKE 'STRING PATTERN WITH HALF WIDTH KATAKANA'.
strings/ctype-mb.c:
A code point of HALF WIDTH KATAKANA at-least need 8 bits.
Promoting the variable from uchar to int will fix the issue.
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
Problem:
If leading zeroes of fractional part of a decimal
number exceeds 45, mod operation on the same fails.
Analysis:
Currently there is a miscalcultion of fractional
part for very small decimals in do_div_mod.
For ex:
For 0.000(45 times).....3
length of the integer part becomes -5 (for a length of one,
buffer can hold 9 digits. Since number of zeroes are 45, integer
part becomes 5) and it is negative because of the leading
zeroes present in the fractional part.
Fractional part is the number of digits present after the
point which is 46 and therefore rounded off to the nearest 9
multiple which is 54. So the length of the resulting fractional
part becomes 6.
Because of this, the combined length of integer part and fractional
part exceeds the max length allocated which is 9 and thereby failing.
Solution:
In case of negative integer value, it indicates there are
leading zeroes in fractional part. As a result stop1 pointer
should be set not just based on frac0 but also intg0. This is
because the detination buffer will be filled with 0's for the length
of intg0.
strings/decimal.c:
Calculate stop1 pointer based on the length of intg0 and frac0.
Bug#18187290 ISSUE WITH BUILDING MYSQL USING CMAKE 2.8.12
We want to upgrade to VS2013 on Windows.
In order to do this, we need to upgrade to cmake 2.8.12
This has introduced some incompatibilities for .pdb files,
and "make install" no longer works.
To reproduce:
cmake --build . --target package --config debug
The fix:
Rather than installing .pdb files for static libraries, we use the /Z7 flag
to store symbolic debugging information in the .obj files.
Description: A typo in create_tailoring() causes the "contraction_flags" to be written
into cs->contractions in the wrong place. This causes two problems:
(1) Anyone relying on `contraction_flags` to decide "could this character be
part of a contraction" is 100% broken.
(2) Anyone relying on `contractions` to determine the weight of a contraction
is mostly broken
Analysis: When we are preparing the contraction in create_tailoring(), we are corrupting the
cs->contractions memory location which is supposed to store the weights(8k) + contraction information(256 bytes). We started storing the contraction information after the 4k location. This is because of logic flaw in the code.
Fix: When we create the contractions, we need to calculate the contraction with (char*) (cs->contractions + 0x40*0x40) from ((char*) cs->contractions) + 0x40*0x40. This makes the "cs->contractions" to move to 8k bytes and stores the contraction information from there. Similarly when we are calculating it for like range queries we need to calculate it from the 8k bytes onwards, this can be done by changing the logic to (const char*) (cs->contractions + 0x40*0x40). And for ucs2 charsets we need to modify the my_cs_can_be_contraction_head() and my_cs_can_be_contraction_tail() to point to 8k+ locations.
RESULTING MY_WC_T RESULT IS NOT USED
Issue : handler functions my_ismbchar_utf8,
my_well_formed_len_mb for charset utf8
is calling unicode converion function
to validate and to find the character
length. Because of this, instructions
which will convert the utf8 to unicode
are executed for no use.
A similar issue exist with charset utf8mb4
Solution : reorganized the code such that character
validation part of unicode conversion
handler is extracted(duplicated) in to
separate function. Hence
my_ismbchar_utf8, my_well_formed_len_mb
will call the new function which only
validates and return the length of mb(utf8).
A similar fix for charset utf8mb4.
strings/ctype-utf8.c:
New functions has been added for charset utf8 and utf8mb4
to validate and to get the length of the character.
Problem:-
We have created a table with UTF8_BIN collation.
In case, when in our query we have ORDER BY clause over a function
call we are getting result in incorrect order.
Note:the bug is not there in 5.5.
Analysis:
In 5.5, for UTF16_BIN, we have min and max multi-byte length is 2 and 4
respectively.In make_sortkey(),for 2 byte character character we are
assuming that the resultant length will be 2 byte/character. But when we
use my_strnxfrm_unicode_full_bin(), we store sorting weights using 3 bytes
per character.This result in truncated result.
Same thing happen for UTF8MB4, where we have 1 byte min multi-byte and
4 byte max multi-byte.We will accsume resultant data as 1 byte/character,
which result in truncated result.
Solution:-
use strnxfrm(means use of MY_CS_STRNXFRM macro) is used for sort, in
which the resultant length is not dependent on source length.
Problem:
=======
It was detected an incorrect behavior of my_strtoll10 function when
converting strings with numbers in the following format:
"184467440XXXXXXXXXYY"
Where XXXXXXXXX > 737095516 and YY <= 15
Samples of problematic numbers:
"18446744073709551915"
"18446744073709552001"
Instead of returning the larger unsigned long long value and setting overflow
in the returned error code, my_strtoll10 function returns the lower 64-bits
of the evaluated number and did not set overflow in the returned error code.
Analysis:
========
Once trying to fix bug 16820156, I've found this bug in the overflow check of
my_strtoll10 function.
This function, once receiving a string with an integer number larger than
18446744073709551615 (the larger unsigned long long number) should return the
larger unsigned long long number and set overflow in the returned error code.
Because of a wrong overflow evaluation, the function didn't catch the
overflow cases where (i == cutoff) && (j > cutoff2) && (k <= cutoff3). When
the overflow evaluation fails, the function return the lower 64-bits of the
evaluated number and do not set overflow in the returned error code.
Fix:
===
Corrected the overflow evaluation in my_strtoll10.
WITH UTF8_UNICODE_CI COLLATION
Problem Description:
When comparing datetime values with strings, the utf8_unicode_ci collation
prevents correct comparisons. Consider the below set of queries, it is not
showing any results on a table which has tuples that satisfies the query.
But for collation utf8_general_ci it shows one tuple.
set names utf8 collate utf8_unicode_ci;;
select * from lang where dt='1979-12-09';
Analysis:
The comparison function is not chosen in case of collation utf8_unicode_ci.
In agg_item_set_converter() because the collation state is having
"MY_CS_NONASCII" for collation type "utf8_unicode_ci". The conversion
of the collation is happening for the date field. And because of that
it is unable to pickup proper compare function(i.e CMP_DATE_WITH_STR).
Actually the bug is accidentally introduced by the WL#3759 in 5.5.
And in 5.6 it is been fixed by the WL#3664.
Fix:
I have backported the changes from the file strings/ctype-uca.c which
are related to "utf8" introduced by the WL#3664.
This change helps in choosing the correct comparison function for all
the collations of utf8 charset.
Bug#12608543: CRASHES WITH DECIMALS AND STATEMENT NEEDS TO BE REPREPARED ERRORS
Backporting these two fixes to 5.1
Added unittest to test my_decimal construtor and assignment operators
sql/my_decimal.h:
Added constructor and assignment operators for my_decimal
unittest/my_decimal/my_decimal-t.cc:
Added test to check constructor and assignment operators for my_decimal
Due to an internal change in the server code in between 5.1 and 5.5
(wl#2649) the hash function used in KEY partitioning changed
for numeric and date/time columns (from binary hash calculation
to character based hash calculation).
Also enum/set changed from latin1 ci based hash calculation to
binary hash between 5.1 and 5.5. (bug#11759782).
These changes makes KEY [sub]partitioned tables on any of
the affected column types incompatible with 5.5 and above,
since the calculation of partition id differs.
Also since InnoDB asserts that a deleted row was previously
read (positioned), the server asserts on delete of a row that
is in the wrong partition.
The solution for this situation is:
1) The partitioning engine will check that delete/update will go to the
partition the row was read from and give an error otherwise, consisting
of the rows partitioning fields. This will avoid asserts in InnoDB and
also alert the user that there is a misplaced row. A detailed error
message will be given, including an entry to the error log consisting
of both table name, partition and row content (PK if exists, otherwise
all partitioning columns).
2) A new optional syntax for KEY () partitioning in 5.5 is allowed:
[SUB]PARTITION BY KEY [ALGORITHM = N] (list_of_cols)
Where N = 1 uses the same hashing as 5.1 (Numeric/date/time fields uses
binary hashing, ENUM/SET uses charset hashing) N = 2 uses the same
hashing as 5.5 (Numeric/date/time fields uses charset hashing,
ENUM/SET uses binary hashing). If not set on CREATE/ALTER it will
default to 2.
This new syntax should probably be ignored by NDB.
3) Since there is a demand for avoiding scanning through the full
table, during upgrade the ALTER TABLE t PARTITION BY ... command is
considered a no-op (only .frm change) if everything except ALGORITHM
is the same and ALGORITHM was not set before, which allows manually
upgrading such table by something like:
ALTER TABLE t PARTITION BY KEY ALGORITHM = 1 () or
ALTER TABLE t PARTITION BY KEY ALGORITHM = 2 ()
4) Enhanced partitioning with CHECK/REPAIR to also check for/repair
misplaced rows. (Also works for ALTER TABLE t CHECK/REPAIR PARTITION)
CHECK FOR UPGRADE:
If the .frm version is < 5.5.3
and uses KEY [sub]partitioning
and an affected column type
then it will fail with an message:
KEY () partitioning changed, please run:
ALTER TABLE `test`.`t1` PARTITION BY KEY ALGORITHM = 1 (a)
PARTITIONS 12
(i.e. current partitioning clause, with the addition of
ALGORITHM = 1)
CHECK without FOR UPGRADE:
if MEDIUM (default) or EXTENDED options are given:
Scan all rows and verify that it is in the correct partition.
Fail for the first misplaced row.
REPAIR:
if default or EXTENDED (i.e. not QUICK/USE_FRM):
Scan all rows and every misplaced row is moved into its correct
partitions.
5) Updated mysqlcheck (called by mysql_upgrade) to handle the
new output from CHECK FOR UPGRADE, to run the ALTER statement
instead of running REPAIR.
This will allow mysql_upgrade (or CHECK TABLE t FOR UPGRADE) to upgrade
a KEY [sub]partitioned table that has any affected field type
and a .frm version < 5.5.3 to ALGORITHM = 1 without rebuild.
Also notice that if the .frm has a version of >= 5.5.3 and ALGORITHM
is not set, it is not possible to know if it consists of rows from
5.1 or 5.5! In these cases I suggest that the user does:
(optional)
LOCK TABLE t WRITE;
SHOW CREATE TABLE t;
(verify that it has no ALGORITHM = N, and to be safe, I would suggest
backing up the .frm file, to be used if one need to change to another
ALGORITHM = N, without needing to rebuild/repair)
ALTER TABLE t <old partitioning clause, but with ALGORITHM = N>;
which should set the ALGORITHM to N (if the table has rows from
5.1 I would suggest N = 1, otherwise N = 2)
CHECK TABLE t;
(here one could use the backed up .frm instead and change to a new N
and run CHECK again and see if it passes)
and if there are misplaced rows:
REPAIR TABLE t;
(optional)
UNLOCK TABLES;
