Commit graph

1533 commits

Author SHA1 Message Date
Oleksandr Byelkin
02e7bff882 Merge commit '10.4' into 10.5 2021-01-06 10:53:00 +01:00
Oleksandr Byelkin
478b83032b Merge branch '10.3' into 10.4 2020-12-25 09:13:28 +01:00
Oleksandr Byelkin
25561435e0 Merge branch '10.2' into 10.3 2020-12-23 19:28:02 +01:00
Sergei Golubchik
f6e91552f0 MDEV-4677 GROUP_CONCAT not showing any output with group_concat_max_len >= 4Gb
don't allow group_concat_max_len values >= 4Gb
(they never worked anyway)
2020-12-10 08:45:20 +01:00
Marko Mäkelä
133b4b46fe Merge 10.4 into 10.5 2020-11-03 16:24:47 +02:00
Marko Mäkelä
533a13af06 Merge 10.3 into 10.4 2020-11-03 14:49:17 +02:00
Marko Mäkelä
c7f322c91f Merge 10.2 into 10.3 2020-11-02 15:48:47 +02:00
Marko Mäkelä
8036d0a359 MDEV-22387: Do not violate __attribute__((nonnull))
This follows up commit
commit 94a520ddbe and
commit 7c5519c12d.

After these changes, the default test suites on a
cmake -DWITH_UBSAN=ON build no longer fail due to passing
null pointers as parameters that are declared to never be null,
but plenty of other runtime errors remain.
2020-11-02 14:19:21 +02:00
Oleksandr Byelkin
48b5777ebd Merge branch '10.4' into 10.5 2020-08-04 17:24:15 +02:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Oleksandr Byelkin
c32f71af7e Merge branch '10.2' into 10.3 2020-08-03 13:41:29 +02:00
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Ian Gilfillan
d2982331a6 Code comment spellfixes 2020-07-22 23:18:12 +02:00
Monty
d1bb7f9143 Fixed memory leak in item_sum.cc::report_cut_value_error()
Only affects DBUG builds
2020-06-24 12:51:55 +03:00
Alexey Botchkov
6c573a9146 MDEV-22844 JSON_ARRAYAGG is limited by group_concat_max_len.
Warning message and function result fixed
2020-06-15 22:51:21 +04:00
Marko Mäkelä
3dbc49f075 Merge 10.4 into 10.5 2020-06-14 10:13:53 +03:00
Marko Mäkelä
805340936a Merge 10.3 into 10.4 2020-06-13 19:01:28 +03:00
Varun Gupta
ab9bd6284c MDEV-22840: JSON_ARRAYAGG gives wrong results with NULL values and ORDER by clause
The problem here is similar to the case with DISTINCT, the tree used for ORDER BY
needs to also hold the null bytes of the record. This was not done for GROUP_CONCAT
as NULLS are rejected by GROUP_CONCAT.

Also introduced a comparator function for the order by tree to handle null
values with JSON_ARRAYAGG.
2020-06-12 23:47:38 +05:30
Varun Gupta
0f6f0daa4d MDEV-22011: DISTINCT with JSON_ARRAYAGG gives wrong results
For DISTINCT to be handled with JSON_ARRAYAGG, we need to make sure
that the Unique tree also holds the NULL bytes of a table record
inside the node of the tree. This behaviour for JSON_ARRAYAGG is
different from GROUP_CONCAT because in GROUP_CONCAT we just reject
NULL values for columns.

Also introduced a comparator function for the unique tree to handle null
values for distinct inside JSON_ARRAYAGG.
2020-06-12 23:47:38 +05:30
Varun Gupta
a006e88cac MDEV-11563: GROUP_CONCAT(DISTINCT ...) may produce a non-distinct list
Backported from MYSQL
Bug : DISTINCT CLAUSE DOES NOT WORK IN GROUP_CONCAT
Issue:
------
The problem occurs when:
1) GROUP_CONCAT (DISTINCT ....) is used in the query.
2) Data size greater than value of system variable:
tmp_table_size.

The result would contain values that are non-unique.

Root cause:
-----------
An in-memory structure is used to filter out non-unique
values. When the data size exceeds tmp_table_size, the
overflow is written to disk as a separate file. The
expectation here is that when all such files are merged,
the full set of unique values can be obtained.

But the Item_func_group_concat::add function is in a bit of
hurry. Even as it is adding values to the tree, it wants to
decide if a value is unique and write it to the result
buffer. This works fine if the configured maximum size is
greater than the size of the data. But since tmp_table_size
is set to a low value, the size of the tree is smaller and
hence requires the creation of multiple copies on disk.

Item_func_group_concat currently has no mechanism to merge
all the copies on disk and then generate the result. This
results in duplicate values.

Solution:
---------
In case of the DISTINCT clause, don't write to the result
buffer immediately. Do the merge and only then put the
unique values in the result buffer. This has be done in
Item_func_group_concat::val_str.

Note regarding result file changes:
-----------------------------------
Earlier when a unique value was seen in
Item_func_group_concat::add, it was dumped to the output.
So result is in the order stored in SE. But with this fix,
we wait until all the data is read and the final set of
unique values are written to output buffer. So the data
appears in the sorted order.

This only fixes the cases when we have DISTINCT without ORDER BY clause
in GROUP_CONCAT.
2020-06-12 23:47:38 +05:30
Varun Gupta
81a08c5462 MDEV-11563: GROUP_CONCAT(DISTINCT ...) may produce a non-distinct list
Backported from MYSQL
 Bug : DISTINCT CLAUSE DOES NOT WORK IN GROUP_CONCAT
    Issue:
    ------
    The problem occurs when:
    1) GROUP_CONCAT (DISTINCT ....) is used in the query.
    2) Data size greater than value of system variable:
    tmp_table_size.

    The result would contain values that are non-unique.

    Root cause:
    -----------
    An in-memory structure is used to filter out non-unique
    values. When the data size exceeds tmp_table_size, the
    overflow is written to disk as a separate file. The
    expectation here is that when all such files are merged,
    the full set of unique values can be obtained.

    But the Item_func_group_concat::add function is in a bit of
    hurry. Even as it is adding values to the tree, it wants to
    decide if a value is unique and write it to the result
    buffer. This works fine if the configured maximum size is
    greater than the size of the data. But since tmp_table_size
    is set to a low value, the size of the tree is smaller and
    hence requires the creation of multiple copies on disk.

    Item_func_group_concat currently has no mechanism to merge
    all the copies on disk and then generate the result. This
    results in duplicate values.

    Solution:
    ---------
    In case of the DISTINCT clause, don't write to the result
    buffer immediately. Do the merge and only then put the
    unique values in the result buffer. This has be done in
    Item_func_group_concat::val_str.

    Note regarding result file changes:
    -----------------------------------
    Earlier when a unique value was seen in
    Item_func_group_concat::add, it was dumped to the output.
    So result is in the order stored in SE. But with this fix,
    we wait until all the data is read and the final set of
    unique values are written to output buffer. So the data
    appears in the sorted order.

    This only fixes the cases when we have DISTINCT without ORDER BY clause
    in GROUP_CONCAT.
2020-06-09 17:55:29 +05:30
Alexey Botchkov
74198384e1 MDEV-21914 JSON_ARRAYAGG doesn't reject ORDER BY clause, but doesn't work either.
ORDER BY fixed for JSON_ARRAYAGG.
2020-06-04 13:53:14 +04:00
Alexey Botchkov
07daf73542 MDEV-22084 Squared brackets missing from JSON_ARRAYAGG when used in a view.
Item_func_groupconcat::print() should be fixed to work for the derived
classes.
2020-06-04 11:00:17 +04:00
Alexey Botchkov
bb47050e1f MDEV-22640, MDEV-22449, MDEV-21528 JSON_ARRAYAGG crashes with NULL values.
We have to include NULL in the result which the GOUP_CONCAT doesn't
always do. Also converting should be done into another String instance
as these can be same.
2020-06-04 10:00:56 +04:00
Marko Mäkelä
23047d3ed4 Merge 10.4 into 10.5 2020-05-18 17:30:02 +03:00
Marko Mäkelä
4f29d776c7 Merge 10.3 into 10.4 2020-05-16 06:27:55 +03:00
Varun Gupta
d49233caf6 MDEV-18100: User defined aggregate functions not working correctly when the schema is changed
The issue here was that when the schema was changed the value for the THD::server_status
is ored with SERVER_SESSION_STATE_CHANGED.
For custom aggregate functions, currently we check if the server_status is equal to
SERVER_STATUS_LAST_ROW_SENT then we should terminate the execution of the custom
aggregate function as there are no more rows to fetch.
So the check should be that if the server status has the bit set for
SERVER_STATUS_LAST_ROW_SENT then we should terminate the execution of the
custom aggregate function.
2020-05-15 15:13:06 +05:30
Marko Mäkelä
8cc15c036d Merge 10.4 into 10.5 2019-12-27 21:17:16 +02:00
Alexander Barkov
4c57ab34d4 Merge remote-tracking branch 'origin/10.3' into 10.4 2019-12-25 13:33:28 +04:00
Alexander Barkov
ee9a19fb05 MDEV-21392 Cleanup redundant overriding in Item_sum_num 2019-12-25 12:23:24 +04:00
Vlad Lesin
6718d3bc32 MDEV-21082: isnan/isinf compilation errors, isfinite warnings on MacOS
The fix consists of three commits backported from 10.3:

1) Cleanup isnan() portability checks

(cherry picked from commit 7ffd7fe962)

2) Cleanup isinf() portability checks

Original problem reported by Wlad: re-compilation of 10.3 on top of 10.2
build would cache undefined HAVE_ISINF from 10.2, whereas it is expected
to be 1 in 10.3.

std::isinf() seem to be available on all supported platforms.
(cherry picked from commit bc469a0bdf)

3) Use std::isfinite in C++ code

This is addition to parent revision fixing build failures.

(cherry picked from commit 54999f4e75)
2019-11-19 16:28:15 +03:00
Alexey Botchkov
ba8e5e689c MDEV-16620 JSON_ARRAYAGG and JSON_OBJECTAGG.
Ison_objectagg implemented.
2019-10-14 14:24:22 +04:00
Alexey Botchkov
7dc74bb3b5 Merge branch 'MDEV-16620' of https://github.com/markus456/server into bb-10.5-hf
Conflicts:
	sql/sql_yacc.yy
2019-10-10 00:31:59 +04:00
Sergei Golubchik
cd41ffe1f1 MDEV-19713 Remove big_tables system variable
mark big_tables deprecated, the server can put temp tables on disk
as needed avoiding "table full" errors.

in case someone would really need to force a tmp table to be created
on disk from the start and for testing allow tmp_memory_table_size
to be set to 0.

fix tests to use that instead (and add a test that it actually
works).

make sure in-memory TREE size limit is never 0 (it's [ab]using
tmp_memory_table_size at the moment)

remove few sys_vars.*_basic tests
2019-09-28 19:21:14 +02:00
Marko Mäkelä
780d2bb8a7 Merge 10.4 into 10.5 2019-09-06 14:25:20 +03:00
Alexander Barkov
52e276247d MDEV-19961 MIN(timestamp_column) returns a wrong result in a GROUP BY query 2019-08-19 15:11:14 +04:00
Alexander Barkov
afe6eb499d Revert "MDEV-20342 Turn Field::flags from a member to a method"
This reverts commit e86010f909.

Reverting on Monty's request, as this change makes merging
things from 10.5 to 10.2 much harder.
2019-08-14 20:27:00 +04:00
Alexander Barkov
e86010f909 MDEV-20342 Turn Field::flags from a member to a method 2019-08-14 13:33:01 +04:00
Alexander Barkov
c1599821a5 Merge remote-tracking branch 'origin/10.4' into 10.5 2019-08-13 23:49:10 +04:00
Marko Mäkelä
624dd71b94 Merge 10.4 into 10.5 2019-08-13 18:57:00 +03:00
Alexander Barkov
95cdc1ca5f Merge commit '43882e764d6867c6855b1ff057758a3f08b25c55' into 10.4 2019-08-13 11:42:31 +04:00
Alexander Barkov
2dac123515 A cleanup for MDEV-20273 Add class Item_sum_min_max - removing duplicate code
Reusing the MIN()/MAX() fix_length_and_dec() related code for window functions
- FIRST_VALUE()
- LAST_VALUE()
- NTH_VALUE()
- LEAD()
- LAG
2019-08-09 09:00:17 +04:00
Alexander Barkov
e978efd96b MDEV-20273 Add class Item_sum_min_max 2019-08-07 14:13:44 +04:00
Marko Mäkelä
e9c1701e11 Merge 10.3 into 10.4 2019-07-25 18:42:06 +03:00
Eugene Kosov
0f83c8878d Merge 10.2 into 10.3 2019-07-16 18:39:21 +03:00
Alexander Barkov
e6ff3f9d1c MDEV-20052 Add a MEM_ROOT pointer argument to Type_handler::make_xxx_field() 2019-07-12 06:58:51 +04:00
Eugene Kosov
26c389b7b7 Merge 10.1 into 10.2 2019-07-09 13:22:22 +03:00
Alexander Barkov
11f13bff21 MDEV-17857 Assertion `tmp != ((long long) 0x8000000000000000LL)' failed in TIME_from_longlong_datetime_packed upon SELECT with GROUP BY 2019-07-05 09:15:40 +04:00
Markus Mäkelä
d0fc07c85f
MDEV-16620: Add JSON_ARRAYAGG function
The JSON_ARRAYAGG function extends the GROUP_CONCAT function and provides
a method of aggregating JSON results. The current implementation supports
DISTINCT and LIMIT but not ORDER BY (Oracle supports GROUP BY).

Adding GROUP BY support is possible but it requires some extra work as the
grouping appears to be done inside a temporary table that complicates
matters.

Added test cases that covert aggregation of all JSON types and JSON
validation for the generated results.
2019-07-04 13:12:08 +03:00
Oleksandr Byelkin
f66d1850ac Merge branch '10.3' into 10.4 2019-06-14 22:10:50 +02:00