mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-29 10:14:19 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Fix for a BUG#31898: 16M memory allocations for user variables

Browse source 
in stored procedure.

The problem was that MySQL used unnecessarily large amounts of
memory if user variables were used as an argument to CONCAT or
CONCAT_WS -- 16M per each user variable used.

Technically, it happened because MySQL used the following
allocation strategy for string functions to avoid multiple
realloc() calls: in the virtual operation fix_length_and_dec()
the attribute max_length was calculated as a sum of max_length
values for each argument.

Although this approach worked well for small (or fixed) data types,
there could be a problem if there as a user variable among
the arguments of a string function -- max_length of the function
would be 16M (as the max_length of a user variable is 16M).

Both CONCAT() and CONCAT_WS() functions suffer from this problem.

The fix is to do not use meta-data for allocating memory.
The following strategy is proposed instead: allocate the exact
length of the result string at the first record, double the amount
of memory allocated when it is required.

No test case for this bug because there is no way to test memory
consumption in a robust way with our test suite.


sql/item_strfunc.cc:
  Implement memory-wise allocation strategy.
This commit is contained in:
unknown 2007-11-12 14:44:17 +03:00
parent c33d42eb32
commit 819eaead10
1 changed files with 55 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

60
sql/item_strfunc.cc
View file

@ -356,10 +356,35 @@ String *Item_func_concat::val_str(String *str)
}
else
{ // Two big const strings
if (tmp_value.alloc(max_length) ||
tmp_value.copy(*res) ||
tmp_value.append(*res2))
/*
NOTE: We should be prudent in the initial allocation unit -- the
size of the arguments is a function of data distribution, which
can be any. Instead of overcommitting at the first row, we grow
the allocated amount by the factor of 2. This ensures that no
more than 25% of memory will be overcommitted on average.
*/
uint concat_len= res->length() + res2->length();
if (tmp_value.alloced_length() < concat_len)
{
if (tmp_value.alloced_length() == 0)
{
if (tmp_value.alloc(concat_len))
goto null;
}
else
{
uint new_len = max(tmp_value.alloced_length() * 2, concat_len);
if (tmp_value.realloc(new_len))
goto null;
}
}
if (tmp_value.copy(*res) || tmp_value.append(*res2))
goto null;
res= &tmp_value;
use_as_buff=str;
}
@ -679,8 +704,33 @@ String *Item_func_concat_ws::val_str(String *str)
}
else
{ // Two big const strings
if (tmp_value.alloc(max_length) ||
tmp_value.copy(*res) ||
/*
NOTE: We should be prudent in the initial allocation unit -- the
size of the arguments is a function of data distribution, which can
be any. Instead of overcommitting at the first row, we grow the
allocated amount by the factor of 2. This ensures that no more than
25% of memory will be overcommitted on average.
*/
uint concat_len= res->length() + sep_str->length() + res2->length();
if (tmp_value.alloced_length() < concat_len)
{
if (tmp_value.alloced_length() == 0)
{
if (tmp_value.alloc(concat_len))
goto null;
}
else
{
uint new_len = max(tmp_value.alloced_length() * 2, concat_len);
if (tmp_value.realloc(new_len))
goto null;
}
}
if (tmp_value.copy(*res) ||
tmp_value.append(*sep_str) ||
tmp_value.append(*res2))
goto null;