Grouping or ordering of long values in not indexed BLOB/TEXT columns
with GBK or BIG5 charsets crashes the server.
MySQL server uses sorting (the filesort procedure) in the temporary
table to evaluate the GROUP BY clause in case of lack of suitable index.
That procedure takes into account only first @max_sort_length bytes
(system variable, usually 1024) of TEXT/BLOB sorting key string.
The my_strnxfrm_gbk and my_strnxfrm_big5 fill temporary keys
with data of whole blob length instead of @max_sort_length bytes
length. That buffer overrun has been fixed.
Problem: "SELECT INTO OUTFILE" created incorrect dumps for BLOBs,
so "LOAD DATA" later incorrectly interpreted 0x5C as the second
byte of a multi-byte sequence, instead of escape character.
Fix: adding escaping of multi-byte heads.
The problem was that when converting a string to an exact number,
rounding didn't work, because conversion didn't understand
approximate numbers notation.
Fix: a new function for string-to-number conversion was implemented,
which is aware of approxinate number notation (with decimal point
and exponent, e.g. -19.55e-1)
into parts when converting to Unicode.
m_ctype.h:
Reorganizing mb_wc return codes to be able
to return "an unassigned N-byte-long character".
sql_string.cc:
Adding code to detect and properly handle
unassigned characters (i.e. the those character
which are correctly formed according to the
character specifications, but don't have Unicode
mapping).
Many files:
Fixing conversion function to return new codes.
ctype_ujis.test, ctype_gbk.test, ctype_big5.test:
Adding a test case.
ctype_ujis.result, ctype_gbk.result, ctype_big5.result:
Fixing results accordingly.
new file
mysql_fix_privilege_tables.sql, mysql_create_system_tables.sh:
Adding true BINARY/VARBINARY: fixing "password" type, not to be 0x00-padding.
Many files:
Adding true BINARY/VARBINARY: fixing tests not to output 0x00 bytes.
Adding true BINARY/VARBINARY: new pad_char structure member.
ctype-bin.c:
Adding true BINARY/VARBINARY: new pad_char structure member.
New strnxfrm, with two trailing length bytes.
field.cc:
Adding true BINARY/VARBINARY.
index doesn't return correct result
item_cmpfunc.cc:
Use charset of LIKE to decide whether
to use 8bit or Unicode "escape" value.
But use charset of "escape" to scan escape character.
strings/ctype-xxx.c:
We cannot reduce "end" pointer using charpos(),
because of possible escape characters in the string.
Limit the loop using count of written characters instead.
ctype_like_escape.inc:
new file
mysql-test/t/ctype_xxx:
mysql-test/r/ctype_xxx:
Adding test case.
In cp932, '\' character can be the second byte in a
multi-byte character stream. This makes it difficult to use
mysql_escape_string. Added flag to indicate which languages allow
'\' as second byte of multibyte sequence so that when putting a prepared
statement into the binlog we can decide at runtime whether hex encoding
is really needed.
ctype-cp932.c:
ctype-gbk.c:
ctype-mb.c:
ctype-simple.c:
ctype-sjis.c:
ctype-ucs2.c:
ctype-ujis.c:
ctype-utf8.c:
Adding explicit cast to return type
in pointer substructions to avoid
warnings from some compilers.
Bug #11987
mysql will truncate the text when the text contain GBK char:"0xA3A0" and "0xA1"
Allow to store and retrieve even unassigned GBK codes.
Like we did in Big5 earlier.
have_gbk.inc, have_gbk.require, ctype_gbk.result, ctype_gbk.test:
new file
New BitKeeper file ``mysql-test/include/ctype_innodb_like.inc''
Many files:
bug#11650: LIKE pattern matching using prefix index doesn't return correct result
min and max values were too long in the case of prefix key.
Fix my_like_range functions not to exceed prefix length.
ctype_innodb_like.inc:
new file
UPPER/LOWER now can return a string with different length.
mi_test1.c:
Adding new arguments.
Many files:
Changeing caseup/casedn to return a result with different
length than argument.
sql_string.h:
Removing unused method,
mysql_priv.h:
Removing unused method
Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART
Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors
Added support for VARCHAR KEYS to heap
Removed support for ISAM
Now only long VARCHAR columns are changed to TEXT on demand (not CHAR)
Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
Don't read character set files if we are using only the default charset. In most cases the user will not anymore get a warning about missing character set files
Compare strings with space extend instead of space strip. Now the following comparisons holds: "a" == "a " and "a\t" < "a". (Bug #3152).
Note: Because of the above fix, one has to do a REPAIR on any table that has an ascii character < 32 last in a CHAR/VARCHAR/TEXT columns.
Added more DBUG statements
Ensure that we are comparing end space with BINARY strings
Use 'any_db' instead of '' to mean any database. (For HANDLER command)
Only strip ' ' when comparing CHAR, not other space-like characters (like \t)
sequence boundaries in functions LIKE and LOCATE in
the case of "binary" collation. Comparison was done
like if the strings were just a binary strings without
character set assumption.
