Commit graph

294 commits

Author SHA1 Message Date
Nayuta Yanagisawa
8c28b27f00 MDEV-28301 Spider: Fix GCC warnings, comparing the result of pointer addition ... and NULL
The condition of the if statements are always true.
2022-05-13 21:32:49 +09:00
Alexander Barkov
9d734cdd61 Merge remote-tracking branch 'origin/10.2' into 10.3 2022-04-14 11:50:34 +04:00
KiyoshiTakeda
4d1955d348
MDEV-28225 Disallow user to create Spider temporary table
Creating a temporary table with Spider is non-sense because a Spider
table cannot hold any physical data and it requires an additional
effort to manage even if it is configured correctly.

Set HTON_TEMPORARY_NOT_SUPPORTED to spider_hton->flags.  

Reviewed-by: nayuta.yanagisawa@hey.com
Co-authored-by: d8sk4ueun@gmail.com
2022-04-11 23:02:38 +09:00
Nayuta Yanagisawa
4194f7b605 MDEV-25116 Spider: IF(COUNT( trigger SQL Error (1054)_ Unknown column '' in field list
The original query "SELECT IF(COUNT(a.`id`)>=0,'Y','N') FROM t" is
transformed to "SELECT COUNT(a.`id`), IF(ref >= 0, 'Y', 'N') FROM t",
where ref is Item_ref to "COUNT(a.`id`)", by split_sum_func().

Spider walks the item list twice, invoking spider_db_print_item_type().
The first invocation is in spider_create_group_by_handler() with
str == NULL. The second one is in spider_group_by_handler::init_scan()
with str != NULL.

spider_db_print_item_type() prints nothing at the first invocation,
and it prints item at the second invocation. However, at the second
invocation, the above mentioned ref to "COUNT(a.`id`)" points to
a field in a temporary table where the result will be stored. Thus,
to look behind the item_ref, Spider need to generate the query earlier.

A possible fix would be to generate a query to send in
spider_create_group_by_handler(). However, the fix requires a
considerable amount of changes of the Spider's GROUP BY handler.
I'd like to avoid that.

So, I fix the problem by not to use the GROUP BY handler when a
query contains Item_ref whose table_name, name, and alias_name_used
are not set.
2022-04-08 15:27:33 +09:00
Alexey Botchkov
466d81709b MDEV-26768 Spider table crashes the server after the mysql_list_fields() client's call and produces weird result for SHOW FIELDS.
Suppress errors in ha_spider::info() called from mysqld_show_fields()
2022-01-20 14:44:56 +04:00
Nayuta Yanagisawa
39f6315612 MDEV-19866 follow-up
Cherry-picking the fix for MDEV-19866 changes the behavior of
the Spider slightly. So, I modified a existing test to match
the new behavior.
2021-10-18 13:19:03 +09:00
Kentoku SHIBA
a46665090b MDEV-19866 With a Spider table, a SELECT with WHERE involving primary key breaks following SELECTs ()
Change checking scanning partitions from part_spec to part_info->read_partitions
2021-10-18 13:19:03 +09:00
Daniel Ye
9fc1ef932f
MDEV-26545 Spider does not correctly handle UDF and stored function in where conds
- Handle stored function conditions correctly, with the same logic as with UDFs.
- When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
- Disable direct update/delete when a udf condition is skipped.
2021-09-22 18:55:05 +09:00
Daniel Ye
ac1c6738f9
MDEV-26545 Spider does not correctly handle UDF and stored function in where conds
- Handle stored function conditions correctly, with the same logic as with UDFs.
- When running queries on Spider SE, by default, we do not push down WHERE conditions containing usage of UDFs/stored functions to remote data nodes, unless the user demands (by setting spider_use_pushdown_udf).
2021-09-22 18:54:22 +09:00
Yongxin Xu
fa6eaead21
MDEV-24523 Execution of JSON_REPLACE failed on Spider
JSON_REPLACE() function executed with an error on Spider SE.
This patch fixes the problem, and it also fixes the MDEV-24541.

The problem is that Item_func_json_insert::func_name() returns
the wrong function name "json_update". 
The Spider SE reconstructs a query based on the return value
in some cases. Thus, if the return value is wrong, the Spider SE
may generate a wrong query.
2021-08-05 12:21:59 +09:00
Yongxin Xu
43099af95b
MDEV-24020: Trim with remove_str Fails on Spider
This patch fixes the bug that TRIM(BOTH ... FROM $str), TRIM(LEADING ... FROM $str),
and TRIM(TRAILING ... FROM $str) failed with errors when executing on Spider.
2021-07-29 00:19:36 +09:00
Nayuta Yanagisawa
f52d39369a MDEV-24517 follow-up: Fix for test with --ps-protocol
Tests for the Spider storage engine often use the following idiom:

--let $command=CREATE TABLE t1 (...);CREATE TABLE t2 (...); ...
--eval $command

However, the idiom seems to work in the normal protocol, but fails
in the prepared statement (ps) protocol.
As testing CREATE TABLE statements in the ps protocol, we wrap the
idiom by --disable_ps_protocol and --enable_ps_protocol.
2021-07-25 18:01:09 +00:00
Yongxin Xu
73d32cc100
MDEV-24517: JSON_EXTRACT as conditions triggers syntax error on Spider ()
The `item_func::JSON_EXTRACT_FUNC` was not handled correctly in the previous
versions on the Spider storage engine, which makes queries like
`SELECT * FROM t1 WHERE json_extract(jdoc, '$.Age')=20`
failed with syntax error.

This patch writes specific code to handle JSON_EXTRACT in the Spider Storage
Engine and fix that bug.
2021-07-23 23:36:27 +09:00
Sergei Golubchik
6190a02f35 Merge branch '10.2' into 10.3 2021-07-21 20:11:07 +02:00
Nayuta Yanagisawa
cf6d83e7d6 MDEV-24760 SELECT..CASE statement syntax error at Spider Engine table
The root cause of the bug is in `spider_db_mbase_util::open_item_func()`.
The function handles an instance of the `Item_func` class based on its
`Item_func::Functype`.
The `Functype` of `CASE WHEN ... THEN` is `CASE_SEARCHED_FUNC`.
However, the Spider SE doesn't recognize this `Functype` because
`CASE_SEARCHED_FUNC` is newly added by 4de0d92. This results in the wrong
handling of `CASE WHEN ... THEN`.

The above also applies to `CASE_SIMPLE_FUNC`.
2021-07-19 15:16:25 +00:00
Nayuta Yanagisawa
0f6e170c34 MDEV-25985 Spider handle ">=" as ">" in some cases
The function spider_db_append_key_where_internal() converts
HA_READ_AFTER_KEY to '>'. The conversion seems to be correct for
single-column indexes because HA_READ_AFTER_KEY means "read the
key after the provided value."

However, how about multi-column indexes? Assume that there is a
multi-column index on c1 and c2 and we search with the condition
'c1 >= 100 AND c2 > 200'. The key_range.flag corresponds to the
search condition could be HA_READ_AFTER_KEY. In such a case,
we could not simply convert HA_READ_AFTER_KEY to '>'.

The correct conversion is to convert HA_READ_AFTER_KEY to '>'
only for the last column in key_part_map and to convert
HA_READ_AFTER_KEY to '>=' for the other column.

The similar discussion also applies to the conversion from
key_range.flag to a sign of inequality.
2021-07-14 12:08:37 +00:00
Sergei Golubchik
068246c006 fix spider tests for --ps
spider tests use

--let $var= many;sql;statements
--eval $var

and this doesn't work in ps
2021-06-19 00:45:49 +02:00
Sergei Golubchik
a5f6eca50d spider tests aren't big
and *never* disable tests in suite.pm based on $::opt_big_test,
this will make the test skipped both as too big (for ./mtr)
and as too small (for ./mtr --big --big).
2021-06-19 00:22:15 +02:00
Nikita Malyavin
21809f9a45 MDEV-17556 Assertion `bitmap_is_set_all(&table->s->all_set)' failed
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.

This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
 bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
 sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
 directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.

The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.

To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
 orig_*, all_set bitmaps are never substituted already.

This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
 to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
 MY_BITMAP* instead of my_bitmap_map*

These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
2021-01-27 00:50:55 +10:00
Kentoku SHIBA
de5e5ab210 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
Add test cases.
2021-01-12 10:25:03 +01:00
Kentoku SHIBA
69c86abb64 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2021-01-12 10:25:03 +01:00
Nikita Malyavin
e25623e78a MDEV-17556 Assertion `bitmap_is_set_all(&table->s->all_set)' failed
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.

This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
 bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
 sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
 directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.

The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.

To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
 orig_*, all_set bitmaps are never substituted already.

This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
 to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
 MY_BITMAP* instead of my_bitmap_map*

These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
2021-01-08 16:04:29 +10:00
Kentoku SHIBA
88d22f0e65 MDEV-20100 MariaDB 13.3.9 Crash "[ERROR] mysqld got signal 11 ;"
Some functions on ha_partition call functions on all partitions, but handler->reset() is only called that pruned by m_partitions_to_reset. So Spider didn't clear pointer on unpruned partitions, if the unpruned partitions are used by next query, Spider reference the pointer that is already freed.
2020-10-20 22:32:12 +09:00
Kentoku SHIBA
9dedba16ab MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to unlock mutex conn->mta_conn_mutex that wasn't locked at storage/spider/spd_db_conn.cc, line 671 2020-09-07 10:18:43 +09:00
Kentoku SHIBA
e976f461d8 MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to unlock mutex conn->mta_conn_mutex that wasn't locked at storage/spider/spd_db_conn.cc, line 671 2020-09-07 10:09:44 +09:00
Kentoku SHIBA
c032c2ef66 MDEV-18993 The keep-alive connection (set spider_conn_recycle_mode = 1) in spider would cause cash in MariaDB ()
Fix the following valgrind error.

==94390== Thread 29:
==94390== Invalid read of size 8
==94390== at 0x78389D: thd_increment_bytes_sent (sql_class.cc:4265)
==94390== by 0xC8EC46: net_real_write (net_serv.cc:730)
==94390== by 0xC8E0C8: net_flush (net_serv.cc:383)
==94390== by 0xC8E4D0: net_write_command (net_serv.cc:521)
==94390== by 0xADCE61: cli_advanced_command (client.c:468)
==94390== by 0xAE3CAF: mysql_close_slow_part (client.c:3671)
==94390== by 0xAE3D28: mysql_close (client.c:3683)
==94390== by 0x149E69A8: spider_db_mbase::disconnect() (spd_db_mysql.cc:2217)
==94390== by 0x1491EA26: spider_db_disconnect(st_spider_conn*) (spd_db_conn.cc:297)
==94390== by 0x14948EBE: spider_free_conn_alloc(st_spider_conn*) (spd_conn.cc:196)
==94390== by 0x1494B26A: spider_free_conn(st_spider_conn*) (spd_conn.cc:1251)
==94390== by 0x1494941F: spider_free_conn_from_trx(st_spider_transaction*, st_spider_conn*, bool, bool, int*) (spd_conn.cc:315)
==94390== Address 0x1f0e0990 is 4,832 bytes inside a block of size 25,728 free'd
==94390== at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==94390== by 0x13F5545: my_free (my_malloc.c:222)
==94390== by 0x6C75B7: ilink::operator delete(void*, unsigned long) (sql_list.h:618)
==94390== by 0x77B9F6: THD::~THD() (sql_class.cc:1724)
==94390== by 0x1494FCE0: spider_bg_conn_action(void*) (spd_conn.cc:2580)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
==94390== Block was alloc'd at
==94390== at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==94390== by 0x13F4DFA: my_malloc (my_malloc.c:101)
==94390== by 0x1491CF06: ilink::operator new(unsigned long) (sql_list.h:614)
==94390== by 0x1494F7FD: spider_bg_conn_action(void*) (spd_conn.cc:2501)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
==94390== Invalid write of size 8
==94390== at 0x7838AF: thd_increment_bytes_sent (sql_class.cc:4265)
==94390== by 0xC8EC46: net_real_write (net_serv.cc:730)
==94390== by 0xC8E0C8: net_flush (net_serv.cc:383)
==94390== by 0xC8E4D0: net_write_command (net_serv.cc:521)
==94390== by 0xADCE61: cli_advanced_command (client.c:468)
==94390== by 0xAE3CAF: mysql_close_slow_part (client.c:3671)
==94390== by 0xAE3D28: mysql_close (client.c:3683)
==94390== by 0x149E69A8: spider_db_mbase::disconnect() (spd_db_mysql.cc:2217)
==94390== by 0x1491EA26: spider_db_disconnect(st_spider_conn*) (spd_db_conn.cc:297)
==94390== by 0x14948EBE: spider_free_conn_alloc(st_spider_conn*) (spd_conn.cc:196)
==94390== by 0x1494B26A: spider_free_conn(st_spider_conn*) (spd_conn.cc:1251)
==94390== by 0x1494941F: spider_free_conn_from_trx(st_spider_transaction*, st_spider_conn*, bool, bool, int*) (spd_conn.cc:315)
==94390== Address 0x1f0e0990 is 4,832 bytes inside a block of size 25,728 free'd
==94390== at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==94390== by 0x13F5545: my_free (my_malloc.c:222)
==94390== by 0x6C75B7: ilink::operator delete(void*, unsigned long) (sql_list.h:618)
==94390== by 0x77B9F6: THD::~THD() (sql_class.cc:1724)
==94390== by 0x1494FCE0: spider_bg_conn_action(void*) (spd_conn.cc:2580)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
==94390== Block was alloc'd at
==94390== at 0x4C29BC3: malloc (vg_replace_malloc.c:299)
==94390== by 0x13F4DFA: my_malloc (my_malloc.c:101)
==94390== by 0x1491CF06: ilink::operator new(unsigned long) (sql_list.h:614)
==94390== by 0x1494F7FD: spider_bg_conn_action(void*) (spd_conn.cc:2501)
==94390== by 0x4E3DDD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==94390== by 0x5FBFEAC: clone (in /usr/lib64/libc-2.17.so)
2020-06-27 14:42:36 +09:00
Kentoku SHIBA
619a2ccd67 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 01:49:46 +09:00
Marko Mäkelä
67f2782413 Fix GCC -Wstringop-truncation 2020-03-30 12:16:07 +03:00
Oleksandr Byelkin
008ee867a4 Merge branch '10.2' into 10.3 2019-12-04 17:46:28 +01:00
Jan Lindström
9d9a2253c6 Merge remote-tracking branch 10.2 into 10.3
Conflicts:
	mysql-test/suite/galera/t/galera_binlog_event_max_size_max-master.opt
	mysql-test/suite/innodb/r/innodb-mdev-7513.result
	mysql-test/suite/innodb/t/innodb-mdev-7513.test
	mysql-test/suite/wsrep/disabled.def
	storage/innobase/ibuf/ibuf0ibuf.cc
2019-12-02 14:35:10 +02:00
willhan
3551cd32a8 MDEV-17508 Fix bug for spider when using "not like"
fix bug for spider where using "not like" ()

test case:
t1 is a spider engine table;
  CREATE TABLE `t1` (
  `id` int(11) NOT NULL DEFAULT '0',
  `name` char(64) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=SPIDER

query: "select * from t1 where name  not like 'x%' " would dispatch "select xxx name name like 'x%' " to remote mysqld, is wrong
2019-11-25 15:40:01 +01:00
Oleksandr Byelkin
896b869685 MDEV-19238 Mariadb spider - crashes on where null
(fix and explanation came with MDEV-20753 (duplicate of this bug))
2019-10-09 08:55:00 +02:00
Vladislav Vaintroub
41290e91b7 Fix CMake warning in spider, in Windows ninja build 2019-09-12 17:06:06 +02:00
Kentoku SHIBA
6626b10a7a
fix for a compiler warning () 2019-08-17 00:40:38 +09:00
Alexander Barkov
e978efd96b MDEV-20273 Add class Item_sum_min_max 2019-08-07 14:13:44 +04:00
Eugene Kosov
26c389b7b7 Merge 10.1 into 10.2 2019-07-09 13:22:22 +03:00
Eugene Kosov
d36c107a6b imporve clang build
cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Debug

Maintainer mode makes all warnings errors. This patch fix warnings. Mostly about
deprecated `register` keyword.

Too much warnings came from Mroonga and I gave up on it.
2019-06-25 13:21:36 +03:00
Marko Mäkelä
d9fe615ef6 spider_db_init(): Do not return uninitialized error_num
If the allocation of spider_table_sts_threads failed,
we would DBUG_RETURN(error_num) without having initialized
it earlier.

Pre-initialize error_num to HA_ERR_OUT_OF_MEM and remove
a lot of assignments that thus became redundant.

This error was introduced in 207594afac
(Spider 3.3.13).
2019-06-14 07:26:51 +03:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Sergei Golubchik
4e1d3f83b7 Merge branch '10.2' into 10.3 2019-03-29 19:41:41 +01:00
Sergei Golubchik
f2a0c758da Merge branch '10.1' into 10.2 2019-03-29 10:58:20 +01:00
Sergei Golubchik
f97d879bf8 cmake: re-enable -Werror in the maintainer mode
now we can afford it. Fix -Werror errors. Note:
* old gcc is bad at detecting uninit variables, disable it.
* time_t is int or long, cast it for printf's
2019-03-27 22:51:37 +01:00
Sergei Golubchik
f38c352172 post-merge: gcc 8 warnings 2019-03-17 13:06:56 +01:00
Kentoku
4e599c74bb MDEV-18313 Supports 'wrapper mariadb' for connection information 2019-01-31 08:51:34 +01:00
Kentoku
470c1b8d56 Fix an error at using spider_direct_sql with temporary table 2019-01-31 08:51:34 +01:00
Kentoku
ea347fcb28 remove unnecessary drop database/table from slave_trx_isolation.test
add simplified slave_trx_isolation.test
2019-01-31 08:51:34 +01:00
Kentoku
be556f817b remove unnecessary drop database/table from quick_mode.test
add simplified quick_mode.test
2019-01-31 08:51:34 +01:00
Kentoku
6d80d35d91 MDEV-16787 optimistic parallel replication fails on spider
Add a system variable spider_slave_trx_isolation.
- spider_slave_trx_isolation
  The transaction isolation level when Spider table is used by slave SQL thread.
  -1 : OFF
   0 : READ UNCOMMITTED
   1 : READ COMMITTED
   2 : REPEATABLE READ
   3 : SERIALIZABLE
  The default value is -1

Miscellaneous Spider typos
2019-01-31 08:51:34 +01:00