Spider converts HA_READ_KEY_EXACT to the equality (=) in the
function spider_db_append_key_where_internal() but the conversion
is not necessarily correct for tables with prefix indices.
We fix the bug by converting HA_READ_KEY_EXACT to 'LIKE "foo%"' when
a target key is a prefix key. The fix is partly inspired by FEDERATED.
See ha_federated::create_where_from_key() for more details.
ha_spider::field_exchange() returns NULL and that results in a crash
or a assertion failure in spider_db_open_item_ident().
In the first place, there seems to be no need to call field_exchange()
for printing an identity (column name with alias). Thus, we simply
remove the call.
The partition storage engine ignores return (error) values of
handler::info(). As a result, a query that should be aborted is
not aborted and then the server violates the assertion.
Item_func_xor and Item_cond are both derived class of Item_bool_func.
Spider converts *Item_func_xor to *Item_cond and then calls the member
function argument_list(), which Item_func_xor does not implement.
Spider supports (or at least allows) INSERT DELAYED but the
documentation does not specify spider as a storage engine that supports
"INSERT DELAYED".
Also, although not mentioned in the documentation, "INSERT DELAYED" is
not intended to be executed inside a transaction, as can be seen from
the list of supported storage engines.
The current implementation allows executing a delayed insert on a
remote transactional table and this breaks the consistency ensured by
the transaction.
We too remove "internal_delayed", one of the Spider table parameters.
Documentation says,
> Whether to transmit existence of delay to remote servers when
> executing an INSERT DELAYED statement on local server.
This table parameter is only used for "INSERT DELAYED".
Reviewed by: Nayuta Yanagisawa
Creating a temporary table with Spider is non-sense because a Spider
table cannot hold any physical data and it requires an additional
effort to manage even if it is configured correctly.
Set HTON_TEMPORARY_NOT_SUPPORTED to spider_hton->flags.
Reviewed-by: nayuta.yanagisawa@hey.com
Co-authored-by: d8sk4ueun@gmail.com
The original query "SELECT IF(COUNT(a.`id`)>=0,'Y','N') FROM t" is
transformed to "SELECT COUNT(a.`id`), IF(ref >= 0, 'Y', 'N') FROM t",
where ref is Item_ref to "COUNT(a.`id`)", by split_sum_func().
Spider walks the item list twice, invoking spider_db_print_item_type().
The first invocation is in spider_create_group_by_handler() with
str == NULL. The second one is in spider_group_by_handler::init_scan()
with str != NULL.
spider_db_print_item_type() prints nothing at the first invocation,
and it prints item at the second invocation. However, at the second
invocation, the above mentioned ref to "COUNT(a.`id`)" points to
a field in a temporary table where the result will be stored. Thus,
to look behind the item_ref, Spider need to generate the query earlier.
A possible fix would be to generate a query to send in
spider_create_group_by_handler(). However, the fix requires a
considerable amount of changes of the Spider's GROUP BY handler.
I'd like to avoid that.
So, I fix the problem by not to use the GROUP BY handler when a
query contains Item_ref whose table_name, name, and alias_name_used
are not set.
- Handle stored function conditions correctly, with the same logic as with UDFs.
- When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
- Disable direct update/delete when a udf condition is skipped.
- Handle stored function conditions correctly, with the same logic as with UDFs.
- When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
JSON_REPLACE() function executed with an error on Spider SE.
This patch fixes the problem, and it also fixes the MDEV-24541.
The problem is that Item_func_json_insert::func_name() returns
the wrong function name "json_update".
The Spider SE reconstructs a query based on the return value
in some cases. Thus, if the return value is wrong, the Spider SE
may generate a wrong query.
This patch fixes the bug that TRIM(BOTH ... FROM $str), TRIM(LEADING ... FROM $str),
and TRIM(TRAILING ... FROM $str) failed with errors when executing on Spider.
Tests for the Spider storage engine often use the following idiom:
--let $command=CREATE TABLE t1 (...);CREATE TABLE t2 (...); ...
--eval $command
However, the idiom seems to work in the normal protocol, but fails
in the prepared statement (ps) protocol.
As testing CREATE TABLE statements in the ps protocol, we wrap the
idiom by --disable_ps_protocol and --enable_ps_protocol.
The `item_func::JSON_EXTRACT_FUNC` was not handled correctly in the previous
versions on the Spider storage engine, which makes queries like
`SELECT * FROM t1 WHERE json_extract(jdoc, '$.Age')=20`
failed with syntax error.
This patch writes specific code to handle JSON_EXTRACT in the Spider Storage
Engine and fix that bug.
The root cause of the bug is in `spider_db_mbase_util::open_item_func()`.
The function handles an instance of the `Item_func` class based on its
`Item_func::Functype`.
The `Functype` of `CASE WHEN ... THEN` is `CASE_SEARCHED_FUNC`.
However, the Spider SE doesn't recognize this `Functype` because
`CASE_SEARCHED_FUNC` is newly added by 4de0d92. This results in the wrong
handling of `CASE WHEN ... THEN`.
The above also applies to `CASE_SIMPLE_FUNC`.
The function spider_db_append_key_where_internal() converts
HA_READ_AFTER_KEY to '>'. The conversion seems to be correct for
single-column indexes because HA_READ_AFTER_KEY means "read the
key after the provided value."
However, how about multi-column indexes? Assume that there is a
multi-column index on c1 and c2 and we search with the condition
'c1 >= 100 AND c2 > 200'. The key_range.flag corresponds to the
search condition could be HA_READ_AFTER_KEY. In such a case,
we could not simply convert HA_READ_AFTER_KEY to '>'.
The correct conversion is to convert HA_READ_AFTER_KEY to '>'
only for the last column in key_part_map and to convert
HA_READ_AFTER_KEY to '>=' for the other column.
The similar discussion also applies to the conversion from
key_range.flag to a sign of inequality.
and *never* disable tests in suite.pm based on $::opt_big_test,
this will make the test skipped both as too big (for ./mtr)
and as too small (for ./mtr --big --big).
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.
This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.
The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.
To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
orig_*, all_set bitmaps are never substituted already.
This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
MY_BITMAP* instead of my_bitmap_map*
These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.
This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.
The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.
To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
orig_*, all_set bitmaps are never substituted already.
This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
MY_BITMAP* instead of my_bitmap_map*
These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
Some functions on ha_partition call functions on all partitions, but handler->reset() is only called that pruned by m_partitions_to_reset. So Spider didn't clear pointer on unpruned partitions, if the unpruned partitions are used by next query, Spider reference the pointer that is already freed.
Fix the following valgrind error.
==94390== Thread 29:
==94390== Invalid read of size 8
==94390== at 0x78389D: thd_increment_bytes_sent (sql_class.cc:4265)
==94390== by 0xC8EC46: net_real_write (net_serv.cc:730)
==94390== by 0xC8E0C8: net_flush (net_serv.cc:383)
==94390== by 0xC8E4D0: net_write_command (net_serv.cc:521)
==94390== by 0xADCE61: cli_advanced_command (client.c:468)
==94390== by 0xAE3CAF: mysql_close_slow_part (client.c:3671)
==94390== by 0xAE3D28: mysql_close (client.c:3683)
==94390== by 0x149E69A8: spider_db_mbase::disconnect() (spd_db_mysql.cc:2217)
==94390== by 0x1491EA26: spider_db_disconnect(st_spider_conn*) (spd_db_conn.cc:297)
==94390== by 0x14948EBE: spider_free_conn_alloc(st_spider_conn*) (spd_conn.cc:196)
==94390== by 0x1494B26A: spider_free_conn(st_spider_conn*) (spd_conn.cc:1251)
==94390== by 0x1494941F: spider_free_conn_from_trx(st_spider_transaction*, st_spider_conn*, bool, bool, int*) (spd_conn.cc:315)
==94390== Address 0x1f0e0990 is 4,832 bytes inside a block of size 25,728 free'd
==94390== at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==94390== by 0x13F5545: my_free (my_malloc.c:222)
==94390== by 0x6C75B7: ilink::operator delete(void*, unsigned long) (sql_list.h:618)
==94390== by 0x77B9F6: THD::~THD() (sql_class.cc:1724)
==94390== by 0x1494FCE0: spider_bg_conn_action(void*) (spd_conn.cc:2580)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
==94390== Block was alloc'd at
==94390== at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==94390== by 0x13F4DFA: my_malloc (my_malloc.c:101)
==94390== by 0x1491CF06: ilink::operator new(unsigned long) (sql_list.h:614)
==94390== by 0x1494F7FD: spider_bg_conn_action(void*) (spd_conn.cc:2501)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
==94390== Invalid write of size 8
==94390== at 0x7838AF: thd_increment_bytes_sent (sql_class.cc:4265)
==94390== by 0xC8EC46: net_real_write (net_serv.cc:730)
==94390== by 0xC8E0C8: net_flush (net_serv.cc:383)
==94390== by 0xC8E4D0: net_write_command (net_serv.cc:521)
==94390== by 0xADCE61: cli_advanced_command (client.c:468)
==94390== by 0xAE3CAF: mysql_close_slow_part (client.c:3671)
==94390== by 0xAE3D28: mysql_close (client.c:3683)
==94390== by 0x149E69A8: spider_db_mbase::disconnect() (spd_db_mysql.cc:2217)
==94390== by 0x1491EA26: spider_db_disconnect(st_spider_conn*) (spd_db_conn.cc:297)
==94390== by 0x14948EBE: spider_free_conn_alloc(st_spider_conn*) (spd_conn.cc:196)
==94390== by 0x1494B26A: spider_free_conn(st_spider_conn*) (spd_conn.cc:1251)
==94390== by 0x1494941F: spider_free_conn_from_trx(st_spider_transaction*, st_spider_conn*, bool, bool, int*) (spd_conn.cc:315)
==94390== Address 0x1f0e0990 is 4,832 bytes inside a block of size 25,728 free'd
==94390== at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==94390== by 0x13F5545: my_free (my_malloc.c:222)
==94390== by 0x6C75B7: ilink::operator delete(void*, unsigned long) (sql_list.h:618)
==94390== by 0x77B9F6: THD::~THD() (sql_class.cc:1724)
==94390== by 0x1494FCE0: spider_bg_conn_action(void*) (spd_conn.cc:2580)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
==94390== Block was alloc'd at
==94390== at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==94390== by 0x13F4DFA: my_malloc (my_malloc.c:101)
==94390== by 0x1491CF06: ilink::operator new(unsigned long) (sql_list.h:614)
==94390== by 0x1494F7FD: spider_bg_conn_action(void*) (spd_conn.cc:2501)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
fix bug for spider where using "not like" (#890)
test case:
t1 is a spider engine table;
CREATE TABLE `t1` (
`id` int(11) NOT NULL DEFAULT '0',
`name` char(64) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=SPIDER
query: "select * from t1 where name not like 'x%' " would dispatch "select xxx name name like 'x%' " to remote mysqld, is wrong
cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Debug
Maintainer mode makes all warnings errors. This patch fix warnings. Mostly about
deprecated `register` keyword.
Too much warnings came from Mroonga and I gave up on it.
If the allocation of spider_table_sts_threads failed,
we would DBUG_RETURN(error_num) without having initialized
it earlier.
Pre-initialize error_num to HA_ERR_OUT_OF_MEM and remove
a lot of assignments that thus became redundant.
This error was introduced in 207594afac
(Spider 3.3.13).
now we can afford it. Fix -Werror errors. Note:
* old gcc is bad at detecting uninit variables, disable it.
* time_t is int or long, cast it for printf's
