Commit graph

71380 commits

Author SHA1 Message Date
Marko Mäkelä
befb0bed68 Merge 10.2 into 10.3 2020-06-08 11:09:49 +03:00
Monty
a9bee9884a Don't allow ALTER TABLE ... ORDER BY on SEQUENCE objects
MDEV-19320 Sequence gets corrupted and produces ER_KEY_NOT_FOUND
           (Can't find record) after ALTER .. ORDER BY
2020-06-07 16:32:00 +03:00
Monty
e6a6382f15 Don't allow illegal create options for SEQUENCE
MDEV-19977 Assertion `(0xFUL & mode) == LOCK_S ||
           (0xFUL & mode) == LOCK_X' failed in lock_rec_lock
2020-06-07 16:32:00 +03:00
Alexander Barkov
fad348a9a6 MDEV-22822 sql_mode="oracle" cannot declare without variable errors 2020-06-07 16:23:47 +04:00
Marko Mäkelä
0e69f601aa Merge 10.4 into 10.5 2020-06-07 12:22:06 +03:00
Sachin
eb14e073ea MDEV-22719 Long unique keys are not created when individual key_part->length < max_key_length but SUM(key_parts->length) > max_key_length
Make UNIQUE HASH key in case when key_info->key_length > max_key_length
2020-06-07 12:07:41 +05:30
Sachin
e208f91ba8 MDEV-21804 Assertion `marked_for_read()' failed upon INSERT into table with long unique blob under binlog_row_image=NOBLOB
Problem:- Calling mark_columns_per_binlog_row_image() earlier may change the
result of mark_virtual_columns_for_write() , Since it can set the bitmap on
for virtual column, and henceforth  mark_virtual_column_deps(field) will
never be called in mark_virtual_column_with_deps.

This bug is not specific for long unique, It also fails for this case
   create table t2(id int primary key, a blob, b varchar(20) as (LEFT(a,2)));
2020-06-07 12:07:36 +05:30
Varun Gupta
d218d1aa49 MDEV-22728: SIGFPE in Unique::get_cost_calc_buff_size from prepare_search_best_index_intersect on optimized builds
For low sort_buffer_size, in the cost calculation of using the Unique object the elements in the tree were evaluated to 0, make sure to have atleast 1 element in the Unique tree.

Also for the function Unique::get allocate memory for atleast MERGEBUFF2+1 keys.
2020-06-07 04:19:58 +05:30
Marko Mäkelä
c7a2fb1e08 Merge 10.3 into 10.4 2020-06-06 22:05:32 +03:00
Igor Babaev
e9dbbf1120 MDEV-22748 MariaDB crash on WITH RECURSIVE large query
This bug is the same as the bug MDEV-17024. The crashes caused by these
bugs were due to premature cleanups of the unit specifying recursive CTEs
that happened in some cases when there were several outer references the
same recursive CTE.
The problem of premature cleanups for recursive CTEs could be already
resolved by the correction in TABLE_LIST::set_as_with_table() introduced
in this patch. ALL other changes introduced by the patches for MDEV-17024
and MDEV-22748 guarantee that this clean-ups are performed as soon as
possible: when the select containing the last outer reference to a
recursive CTE is being cleaned up the specification of the recursive CTE
should be cleaned up as well.
2020-06-06 11:56:10 -07:00
Marko Mäkelä
b3e395a13e Merge 10.2 into 10.3 2020-06-06 18:50:25 +03:00
Marko Mäkelä
0df01ccb66 Merge 10.1 into 10.2 2020-06-06 18:07:04 +03:00
Alexander Barkov
79cdd7e76b MDEV-20305 Data loss on DOUBLE and DECIMAL conversion to INT
Bit operators (~ ^ | & << >>) and the function BIT_COUNT()
always called val_int() for their arguments.
It worked correctly only for INT type arguments.

In case of DECIMAL and DOUBLE arguments it did not work well:
the argument values were truncated to the maximum SIGNED BIGINT value
of 9223372036854775807.

Fixing the code as follows:

- If the argument if of an integer data type,
  it works using val_int() as before.

- If the argument if of some other data type, it gets the argument value
  using val_decimal(), to avoid truncation, and then converts the result
  to ulonglong.

Using Item_handled_func to switch between the two approaches easier.

As an additional advantage, with Item_handled_func it will be easier
to implement overloading in the future, so data type plugings will be able
to define their own behavioir of bit operators and BIT_COUNT().

Moving the code from the former val_int() implementations
as methods to Longlong_null, to avoid code duplication in the
INT and DECIMAL branches.
2020-06-06 11:33:11 +04:00
Igor Babaev
a8c200c73c MDEV-22042 Server crash in Item_field::print on ANALYZE FORMAT=JSON
When processing a query with a recursive CTE a temporary table is used for
each recursive reference of the CTE. As any temporary table it uses its own
mem-root for table definition structures. Due to specifics of the current
implementation of ANALYZE stmt command this mem-root can be freed only at
the very of query processing. Such deallocation of mem-root memory happens
in close_thread_tables(). The function looks through the list of the tmp
tables rec_tables attached to the THD of the query and frees corresponding
mem-roots. If the query uses a stored function then such list is created
for each query of the function. When a new rec_list has to be created the
old one has to be saved and then restored at the proper moment.
The bug occurred because only one rec_list for the query containing CTE was
created. As a result close_thread_tables() freed tmp mem-roots used for
rec_tables prematurely destroying some data needed for the output produced
by the ANALYZE command.
2020-06-05 11:00:07 -07:00
Marko Mäkelä
6877ef9a7c Merge 10.4 into 10.5 2020-06-05 20:36:43 +03:00
Julius Goryavsky
5f55f69e4a Merge 10.1 into 10.2 2020-06-05 18:32:37 +02:00
Marko Mäkelä
68d9d512e9 Merge 10.3 into 10.4 2020-06-05 18:05:22 +03:00
Marko Mäkelä
680463a8d9 Merge 10.2 into 10.3 2020-06-05 16:51:26 +03:00
Thirunarayanan Balathandayuthapani
de1dbb7180 MDEV-21282 Assertion 'mariadb_table' failed in gcol.innodb_virtual_debug_purge
- commit ea37b14409 (MDEV-16678) caused
a regression. when purge thread tries to open the table for virtual
column computation, there is no need to acquire MDL for the table.
Because purge thread already hold MDL for the table
2020-06-05 19:13:44 +05:30
Kentoku SHIBA
23c8adda74 MDEV-6268 SPIDER table with no COMMENT clause causes queries to wait forever
Add looping check

Conflicts:
	sql/table.h
2020-06-05 17:29:59 +09:00
Kentoku SHIBA
e954d9de88 MDEV-19002 Spider performance optimization with partition
Change the following function for batch call instead of each partition
- store_lock
- external_lock
- start_stmt
- extra
- cond_push
- info_push
- top_table
2020-06-05 17:29:57 +09:00
Nikita Malyavin
8e6e5acef1 MDEV-22753 Server crashes upon INSERT into versioned partitioned table with WITHOUT OVERLAPS
Add `append_system_key_parts` call inside `fast_alter_partition_table` during new partition creation.
2020-06-05 20:04:37 +10:00
Nikita Malyavin
35d327fddb MDEV-22599 WITHOUT OVERLAPS does not work with prefix indexes
cmp_max is used instead of cmp to compare key_parts
2020-06-05 20:04:37 +10:00
Nikita Malyavin
0c595bdeaa MDEV-22434 UPDATE on RocksDB table with WITHOUT OVERLAPS fails
Insert worked incorrect as well. RocksDB used table->record[0] internally to store some
intermediate results for key conversion, during index searching among other operations.
So table->record[0] is spoiled during ha_rnd_index_map in ha_check_overlaps, so in turn
the broken record data was inserted.

The fix is to store RocksDB intermediate result in its own buffer instead of table->record[0].

`rocksdb` MTR suite is is checked and runs fine.
No need for additional tests. The existing overlaps.test covers the case completely.
However, I am not going to add anything related to rocksdb to suite, to keep it away
from additional dependencies.

To run tests with RocksDB engine, one can add following to engines.combinations:
[rocksdb]
plugin-load=$HA_ROCKSDB_SO
default-storage-engine=rocksdb
rocksdb
2020-06-05 20:04:37 +10:00
Sergey Vojtovich
dce4c0f979 MDEV-22339 - Assertion `str_length < len' failed
When acquiring SNW/SNRW/X MDL lock DDL/admin statements may abort pending
thr lock in concurrent connection with open HANDLER (or delayed insert
thread).

This may lead to a race condition when table->alias is accessed
concurrently by such threads. Either assertion failure or memory leak
is a practical consequence of this race condition.

Specifically HANDLER is opening a table and issuing alias.copy(), while
DDL executing get_lock_data()/alias.c_ptr()/realloc()/realloc_raw().

Fixed by perforimg table->init() before it is published via
thd->open_tables.
2020-06-04 23:52:10 +02:00
Varun Gupta
f30ff10c8d MDEV-22715: SIGSEGV in radixsort_for_str_ptr and in native_compare/my_qsort2 (optimized builds)
For DECIMAL[(M[,D])] datatype max_sort_length was not being honoured which was leading to buffer
overflow while making the sort key. The fix to this problem would be to create sort keys for decimals
with atmost max_sort_key bytes

Important:
The minimum value of max_sort_length has been raised to 8 (previously was 4),
so fixed size datatypes like DOUBLE and BIGINIT are not truncated for
lower values of max_sort_length.
2020-06-05 01:11:03 +05:30
Varun Gupta
6404645980 MDEV-21626: Optimizer misses the details about the picked join order
Added cost of sorting estimate to the optimizer trace
2020-06-04 20:03:22 +05:30
Varun Gupta
f69278bcd0 MDEV-16230: Server crashes when Analyze format=json is run with a window function with empty PARTITION BY and ORDER BY clauses
Currently when both PARTITION BY and ORDER BY clauses are empty then we create a Item
with the first field in the select list and sort with that field.
It should be created as an Item_temptable_field instead of Item_field because the
print() function continues to work even if the table has been dropped.
2020-06-04 17:03:03 +05:30
Alexey Botchkov
2fcff310d0 MDEV-21902 Nested JSON_ARRAYAGG in JSON_OBJECT should not get escaped. 2020-06-04 14:21:04 +04:00
Alexey Botchkov
74198384e1 MDEV-21914 JSON_ARRAYAGG doesn't reject ORDER BY clause, but doesn't work either.
ORDER BY fixed for JSON_ARRAYAGG.
2020-06-04 13:53:14 +04:00
Aleksey Midenkov
05693cf214 MDEV-22112 Assertion `tab_part_info->part_type == RANGE_PARTITION || tab_part_info->part_type == LIST_PARTITION' failed in prep_alter_part_table
Incorrect syntax for SYSTEM_TIME partition. work_part_info is detected
as HASH partition. We cannot add partition of different type neither
we cannot reorganize SYSTEM_TIME into/from different type
partitioning.

The sidefix for version until 10.5 corrects the message:
"For LIST partitions each partition must be defined"
2020-06-04 12:12:49 +03:00
Alexey Botchkov
07daf73542 MDEV-22084 Squared brackets missing from JSON_ARRAYAGG when used in a view.
Item_func_groupconcat::print() should be fixed to work for the derived
classes.
2020-06-04 11:00:17 +04:00
Alexey Botchkov
bb47050e1f MDEV-22640, MDEV-22449, MDEV-21528 JSON_ARRAYAGG crashes with NULL values.
We have to include NULL in the result which the GOUP_CONCAT doesn't
always do. Also converting should be done into another String instance
as these can be same.
2020-06-04 10:00:56 +04:00
sjaakola
8ec0e9111a MDEV-22763 backporting MDEV-20225 fix into 10.1
Backported the support for aborting and replaying stored procedure and fix for trigger
key assigments from 10.4 version.
Backported also two mtr tests: wsrep_sp_bf_abort and MDEV-20225
2020-06-03 15:34:44 +02:00
Marko Mäkelä
701efbb25b Merge 10.4 into 10.5 2020-06-03 09:45:39 +03:00
Marko Mäkelä
8059148154 Merge 10.3 into 10.4 2020-06-03 07:32:09 +03:00
Varun Gupta
d5e8b4d7f9 MDEV-22509: Server crashes in Field_inet6::store_inet6_null_with_warn / Field::maybe_null
For field with type INET, during EITS collection the min and max values are store in text
representation in the statistical table.
While retrieving the value from the statistical table, the value is stored back in the original
field using binary form instead of text and this was resulting in the crash.

Introduced 2 functions in the Field structure:
  1) store_to_statistical_minmax_field
  2) store_from_statistical_minmax_field
2020-06-02 17:43:45 +05:30
Marko Mäkelä
8300f639a1 Merge 10.2 into 10.3 2020-06-02 10:25:11 +03:00
Varun Gupta
ade8253cb9 MDEV-22303: Incorrect ordering with REGEXP_REPLACE and OFFSET/LIMIT
For character sets and collation where character to weight mapping > 1,
there we need to make sure while creating a sort key,
a temporary buffer is created to store the value of the item by val_str function
and then copy that value back to the sort buffer.
In this case when using a priority queue Sort_param::tmp_buffer was not allocated.

Minor refactoring:
Changed Sort_param::tmp_buffer from char* to String
2020-06-01 17:53:07 +05:30
Marko Mäkelä
d72eebaa3d Merge 10.1 into 10.2 2020-06-01 09:33:03 +03:00
Alexander Barkov
33b839b2e7 MDEV-20280 PERCENTILE_DISC() rejects temporal and string input 2020-06-01 14:30:21 +04:00
Alexander Barkov
f22093ad39 MDEV-22764 Crash with a stored aggregate function returning INET6
Item_sum_sp did not override val_native(). So the reported script
crashed in the default implementation in Item::val_native() on DBUG_ASSERT().

Implementing a correct Item_sum_sp::val_native().
2020-06-01 14:02:31 +04:00
Alexander Barkov
35cbbd4d70 MDEV-20809 EXTRACT from INET6 value does not produce any warnings
Disallowing EXTRACT(xxx FROM inet6arg) as fix time.
Adding a new method Type_handler::can_return_extract_source().
2020-06-01 10:35:01 +04:00
Marko Mäkelä
4a0b56f604 Merge 10.4 into 10.5 2020-05-31 10:28:59 +03:00
Alexander Barkov
0bf843cd13 MDEV-20366 Server crashes in get_current_user upon SET PASSWORD via SP
The opt_for_user subrule was incorrectly scanned before sp_create_assignment_lex(),
so the user name and the host were created on a wrong memory root.

- Reoganizing the grammar to make sure that sp_create_assignment_lex()
  is called immediately after PASSWORD_SYM is scanned, so all attributes
  are then allocated on its memory root.

- Moving the semantic code as methods to LEX, so the grammar looks as simple as possible.

- Changing text_or_password to be of the data type USER_AUTH*.
  As a side effect, the LEX::definer member is now not used when processing
  the SET PASSWORD statement. Everything is done using Bison's stack.

The bug sas introduced by this commit:
commit bf5a144e16
2020-05-30 14:00:56 +04:00
Marko Mäkelä
6da14d7b4a Merge 10.3 into 10.4 2020-05-30 11:04:27 +03:00
Monty
df4ab26a6b SHOW TABLE STATUS now shows if an Aria table is transactional or not
This change also affects information_schema.tables

The create table option "transactional=0 | 1" is now always shown for
storage engines that supports both transactional/crash safe tables and
non transactional tables.

Before this patch the transactional=... option was only shown if the user
specified transactional=... in the CREATE TABLE or ALTER TABLE statement.
The reason for the change was to be able to make it easy to know if an Aria
table is transactional or not.
2020-05-29 22:47:37 +03:00
Marko Mäkelä
e9aaa10c11 Merge 10.2 into 10.3 2020-05-29 22:21:19 +03:00
Sergey Vojtovich
c279878493 Thread safe histograms loading
Previously multiple threads were allowed to load histograms concurrently.
There were no known problems caused by this. But given amount of data
races in this code, it'd happen sooner or later.

To avoid scalability bottleneck, histograms loading is protected by
per-TABLE_SHARE atomic variable.

Whenever histograms were loaded by preceding statement (hot-path), a
scalable load-acquire check is performed.

Whenever histograms have to be loaded anew, mutual exclusion for loaders
is established by atomic variable. If histograms are being loaded
concurrently, statement waits until load is completed.

- Table_statistics::total_hist_size moved to TABLE_STATISTICS_CB: only
  meaningful within TABLE_SHARE (not used for collected stats).
- TABLE_STATISTICS_CB::histograms_can_be_read and
  TABLE_STATISTICS_CB::histograms_are_read are replaced with a tri state
  atomic variable.
- Simplified away alloc_histograms_for_table_share().

Note: there's still likely a data race if a thread attempts accessing
histograms data after it failed to load it (because of concurrent load).
It was there previously and goes out of the scope of this effort. One way
of fixing it could be reviving TABLE::histograms_are_read and adding
appropriate checks whenever it is needed.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected
2020-05-29 21:53:54 +04:00
Sergey Vojtovich
609a0d3db3 Thread safe statistics loading
Previously multiple threads were allowed to load statistics concurrently.
There were no known problems caused by this. But given amount of data
races in this code, it'd happen sooner or later.

To avoid scalability bottleneck, statistics loading is protected by
per-TABLE_SHARE atomic variable.

Whenever statistics were loaded by preceding statement (hot-path), a
scalable load-acquire check is performed.

Whenever statistics have to be loaded anew, mutual exclusion for loaders
is established by atomic variable. If statistics are being loaded
concurrently, statement waits until load is completed.

TABLE_STATISTICS_CB::stats_can_be_read and
TABLE_STATISTICS_CB::stats_is_read are replaced with a tri state atomic
variable.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected
2020-05-29 21:53:54 +04:00