Fix for bug#57272: crash in rpad() when using utf8

Problem: if multibyte and binary string arguments passed to 
RPAD(), LPAD() or INSERT() functions, they might return 
wrong results or even lead to a server crash due to missed
character set convertion.

Fix: perform the convertion if necessary.


mysql-test/r/ctype_utf8.result:
  Fix for bug#57272: crash in rpad() when using utf8
    - test result.
mysql-test/t/ctype_utf8.test:
  Fix for bug#57272: crash in rpad() when using utf8
    - test case.
sql/item_strfunc.cc:
  Fix for bug#57272: crash in rpad() when using utf8
    - convert multibyte argument's character set to binary in case of
  FUNCTION(MULTIBYTE_ARG, .., BINARY_ARG,..) for RPAD(), LPAD() and 
  INSERT() functions.
This commit is contained in:
Ramil Kalimullin 2010-10-12 23:25:40 +04:00
commit 42550e21e8
3 changed files with 93 additions and 0 deletions

View file

@ -1013,6 +1013,20 @@ String *Item_func_insert::val_str(String *str)
if ((length < 0) || (length > res->length()))
length= res->length();
/*
There is one exception not handled (intentionaly) by the character set
aggregation code. If one string is strong side and is binary, and
another one is weak side and is a multi-byte character string,
then we need to operate on the second string in terms on bytes when
calling ::numchars() and ::charpos(), rather than in terms of characters.
Lets substitute its character set to binary.
*/
if (collation.collation == &my_charset_bin)
{
res->set_charset(&my_charset_bin);
res2->set_charset(&my_charset_bin);
}
/* start and length are now sufficiently valid to pass to charpos function */
start= res->charpos((int) start);
length= res->charpos((int) length, (uint32) start);
@ -2514,6 +2528,20 @@ String *Item_func_rpad::val_str(String *str)
/* Set here so that rest of code sees out-of-bound value as such. */
if ((ulonglong) count > INT_MAX32)
count= INT_MAX32;
/*
There is one exception not handled (intentionaly) by the character set
aggregation code. If one string is strong side and is binary, and
another one is weak side and is a multi-byte character string,
then we need to operate on the second string in terms on bytes when
calling ::numchars() and ::charpos(), rather than in terms of characters.
Lets substitute its character set to binary.
*/
if (collation.collation == &my_charset_bin)
{
res->set_charset(&my_charset_bin);
rpad->set_charset(&my_charset_bin);
}
if (count <= (res_char_length= res->numchars()))
{ // String to pad is big enough
res->length(res->charpos((int) count)); // Shorten result if longer
@ -2616,6 +2644,20 @@ String *Item_func_lpad::val_str(String *str)
if ((ulonglong) count > INT_MAX32)
count= INT_MAX32;
/*
There is one exception not handled (intentionaly) by the character set
aggregation code. If one string is strong side and is binary, and
another one is weak side and is a multi-byte character string,
then we need to operate on the second string in terms on bytes when
calling ::numchars() and ::charpos(), rather than in terms of characters.
Lets substitute its character set to binary.
*/
if (collation.collation == &my_charset_bin)
{
res->set_charset(&my_charset_bin);
pad->set_charset(&my_charset_bin);
}
res_char_length= res->numchars();
if (count <= res_char_length)