When using GROUP_CONCAT with ORDER BY, a tree is used for the sorting, as
opposed to normal nested loops join used when there is no ORDER BY.
The tree traversal that generates the result counts the lines that have been
cut down. (as they get cut down to the field's max_size)
But the check of that count was before the tree traversal, so no
warning was generated if the output is truncated.
Fixed by moving the check to after the tree traversal.
When creating a temporary table the concise column type
of a string expression is decided based on its length:
- if its length is under 512 it is stored as either
varchar or char.
- otherwise it is stored as a BLOB.
There is a flag (convert_blob_length) to create_tmp_field
that, when >0 allows to force creation of a varchar if the
max blob length is under convert_blob_length.
However it must be verified that convert_blob_length
(settable through a SQL option in some cases) is
under the maximum that can be stored in a varchar column.
While performing that check for expressions in
create_tmp_field_from_item the max length of the blob was
used instead. This causes blob columns to be created in the
heap temp table used by GROUP_CONCAT (where blobs must not
be created in the temp table because of the constant
convert_blob_length that is passed to create_tmp_field() ).
And since these blob columns are not expected in that place
we get wrong results.
Fixed by checking that the value of the flag variable is
in the limits that fit into VARCHAR instead of the max length
of the blob column.
Problem: GROUP_CONCAT on a multi-byte column can truncate
in the middle of a multibyte character when applying
group_concat_max_len limit. It produces an invalid
multi-byte character in the result string.
The second, easier version - reusing old "warning_for_row" flag,
instead of introducing of "result_is_full" - which was
added in the previous commit.
When resolving unqualified name references MySQL was not
checking what is the item type for the reference. Thus
e.g a string literal item that has by convention a name
equal to its string value will also work as a reference to
a SELECT list item or a table field.
Fixed by allowing only Item_ref or Item_field to referenced by
(unqualified) name.
statement.
The problem was that during statement re-execution if the result was
empty the old result could be returned for group functions.
The solution is to implement proper cleanup() method in group
functions.
that returns the results of aggregation by GROUP_CONCAT.
The crash was due to an overflow happened for the field
sortoder->length.
The fix prevents this overflow exploiting the fact that the
value of sortoder->length cannot be greater than the value of
thd->variables.max_sort_length.
when calculating GROUP_CONCAT all blob fields are transformed
to varchar when making the temp table.
However a varchar has at max 2 bytes for length.
This fix makes the conversion only for blobs whose max length
is below that limit.
Otherwise blob field is created by make_string_field() call.
used
In a simple queries a result of the GROUP_CONCAT() function was always of
varchar type.
But if length of GROUP_CONCAT() result is greater than 512 chars and temporary
table is used during select then the result is converted to blob, due to
policy to not to store fields longer than 512 chars in tmp table as varchar
fields.
In order to provide consistent behaviour, result of GROUP_CONCAT() now
will always be converted to blob if it is longer than 512 chars.
Item_func_group_concat::field_type() is modified accordingly.
The GROUP_CONCAT uses its own temporary table. When ROLLUP is present
it creates the second copy of Item_func_group_concat. This copy receives the
same list of arguments that original group_concat does. When the copy is
set up the result_fields of functions from the argument list are reset to the
temporary table of this copy.
As a result of this action data from functions flow directly to the ROLLUP copy
and the original group_concat functions shows wrong result.
Since queries with COUNT(DISTINCT ...) use temporary tables to store
the results the COUNT function they are also affected by this bug.
The idea of the fix is to copy content of the result_field for the function
under GROUP_CONCAT/COUNT from the first temporary table to the second one,
rather than setting result_field to point to the second temporary table.
To achieve this goal force_copy_fields flag is added to Item_func_group_concat
and Item_sum_count_distinct classes. This flag is initialized to 0 and set to 1
into the make_unique() member function of both classes.
To the TMP_TABLE_PARAM structure is modified to include the similar flag as
well.
The create_tmp_table() function passes that flag to create_tmp_field().
When the flag is set the create_tmp_field() function will set result_field
as a source field and will not reset that result field to newly created
field for Item_func_result_field and its descendants. Due to this there
will be created copy func to copy data from old result_field to newly
created field.
Added test cases for bug #12863.
item_sum.cc, item_sum.h:
Fixed bug #12863.
Added a flag to Item_func_group_concat set to FALSE after
concatenation of the first element of a group.
Item_func_group_concat::fix_fields() set maybe_null flag to 0, and set it to
1 only if some of it's arguments may be null. When used in subquery in tmp
table created field which can't be null. When no data retireved result field
have to be set to null and error mentioned in bug report occurs. Also this
bug can occur if selecting from not null field in empty table.
Function group_concat now marked maybe_null from the very beginning not only
if some of it's argument may be null.
Cannot convert the charset of a GROUP_CONCAT result:
item_sum.cc:
"result" character set was not set into proper value.
func_gconcat.result, func_gconcat.test:
Fixing tests accordingly.
Added a test case for bug #12095.
sql_class.h:
Fixed bug #12095: a join query with GROUP_CONCAT over a single row table.
Added a flag to the TMP_TABLE_PARAM class forcing to put constant
items generated after elimination of a single row table into temp table
in some cases (e.g. when GROUP_CONCAT is calculated over a single row
table).
bk ci sql/item_sum.cc
Fixed bug #12095: a join query with GROUP_CONCAT over a single row table.
If GROUP_CONCAT is calculated we always put its argument into a temp
table, even when the argument is a constant item.
sql_select.cc:
Fixed bug #12095: a join query with GROUP_CONCAT over one row table.
If temp table is used to calculate GROUP_CONCAT the argument should
be always put into this table, even when it is a constant item.
Adding test
item_sum.cc:
Adding a call for collation/charset aggregation,
to collect attributes from the arguments. The actual bug fix.
item_func.h, item_func.cc, item.h, item.cc:
- Removing collation aggrgation functions from Item_func class
in item.cc, and adding it as non-class functions in item.cc
to be able to reuse this code for group_concat.
- Adding replacement for these functions into Item_func class
as wrappers for moved functions, to minizize patch size,
Ensure that 'null_value' is not accessed before val() is called in FIELD() functions
Fixed initialization of key maps. This fixes some problems with keys when you have more than 64 keys
Fixed that ROLLUP don't always create a temporary table. This fix ensures that func_gconcat.test results are now predictable
Added a test case for bug #7769.
item_sum.h:
Fixed bug #7769: a crash for queries with group_concat and
having when the query table was empty.
The bug was due an unsafe dereferencing.
(Bug #4315: GROUP_CONCAT with ORDER BY returns strange results for TEXT fields
Bug #5564: Strange behaviour with group_concat and distinct
Bug #5970: group_concat doesn't print warnings)
