Commit graph

116 commits

Author SHA1 Message Date
Sergei Golubchik
25d9d2e37f Merge branch 'bb-10.4-release' into bb-10.5-release 2021-02-15 16:43:15 +01:00
Sergei Golubchik
00a313ecf3 Merge branch 'bb-10.3-release' into bb-10.4-release
Note, the fix for "MDEV-23328 Server hang due to Galera lock conflict resolution"
was null-merged. 10.4 version of the fix is coming up separately
2021-02-12 17:44:22 +01:00
Nikita Malyavin
21809f9a45 MDEV-17556 Assertion `bitmap_is_set_all(&table->s->all_set)' failed
The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.

This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
 bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
 sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
 directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.

The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.

To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
 orig_*, all_set bitmaps are never substituted already.

This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
 to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
 MY_BITMAP* instead of my_bitmap_map*

These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.
2021-01-27 00:50:55 +10:00
Kentoku SHIBA
69c86abb64 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2021-01-12 10:25:03 +01:00
Kentoku SHIBA
e976f461d8 MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to unlock mutex conn->mta_conn_mutex that wasn't locked at storage/spider/spd_db_conn.cc, line 671 2020-09-07 10:09:44 +09:00
Kentoku SHIBA
9b68847127 MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to unlock mutex conn->mta_conn_mutex that wasn't locked at storage/spider/spd_db_conn.cc, line 671 2020-09-07 10:07:55 +09:00
Kentoku SHIBA
2cc9e45693 MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to unlock mutex conn->mta_conn_mutex that wasn't locked at storage/spider/spd_db_conn.cc, line 671 2020-09-07 09:56:52 +09:00
Kentoku SHIBA
8f8f2aea93 MDEV-23561 Spider doesn't work with ps protocol 2020-08-26 06:54:00 +09:00
Kentoku SHIBA
2000d05c2e MDEV-22246 Result rows duplicated by spider engine
fix the following type mrr scan
(select 0,`id`,`node` from `auto_test_remote`.`tbl_a` where (`id` <> 0) order by `id`)union all(select 1,`id`,`node` from `auto_test_remote`.`tbl_a` where (`id` <> 0) order by `id`) order by `id`
2020-08-24 06:12:15 +09:00
Kentoku SHIBA
65ee216c35 MDEV-22246 Result rows duplicated by spider engine
fix the following type mrr scan
(select 0,`id`,`node` from `auto_test_remote`.`tbl_a` where (`id` <> 0) order by `id`)union all(select 1,`id`,`node` from `auto_test_remote`.`tbl_a` where (`id` <> 0) order by `id`) order by `id`
2020-08-24 06:11:01 +09:00
Kentoku SHIBA
bd62a636a4 fix a compiler warning on GCC 9.3.0 2020-06-14 18:40:07 +09:00
Kentoku SHIBA
a756d54704 Fix issue caused by using spider_bgs_mode = 2 when Spider use limit_mode = 1 internally for data nodes. 2020-06-05 17:30:03 +09:00
Kentoku SHIBA
bbb1140d47 add pointer of ha_spider to Spider's use_result function 2020-06-05 17:30:02 +09:00
Kentoku SHIBA
94861b83f4 prepare for adding new connectors for Spider
Conflicts:
	storage/spider/spd_conn.cc
2020-06-05 17:30:00 +09:00
Kentoku SHIBA
23c8adda74 MDEV-6268 SPIDER table with no COMMENT clause causes queries to wait forever
Add looping check

Conflicts:
	sql/table.h
2020-06-05 17:29:59 +09:00
Kentoku SHIBA
272625d92a fix Spider executing one by one commands before SQL 2020-06-05 17:29:59 +09:00
Kentoku SHIBA
e954d9de88 MDEV-19002 Spider performance optimization with partition
Change the following function for batch call instead of each partition
- store_lock
- external_lock
- start_stmt
- extra
- cond_push
- info_push
- top_table
2020-06-05 17:29:57 +09:00
Monty
4102f1589c Aria will now register it's transactions
MDEV-22531 Remove maria::implicit_commit()
MDEV-22607 Assertion `ha_info->ht() != binlog_hton' failed in
           MYSQL_BIN_LOG::unlog_xa_prepare

From the handler point of view, Aria now looks like a transactional
engine. One effect of this is that we don't need to call
maria::implicit_commit() anymore.

This change also forces the server to call trans_commit_stmt() after doing
any read or writes to system tables.  This work will also make it easier
to later allow users to have system tables in other engines than Aria.

To handle the case that Aria doesn't support rollback, a new
handlerton flag, HTON_NO_ROLLBACK, was added to engines that has
transactions without rollback (for the moment only binlog and Aria).

Other things
- Moved freeing of MARIA_SHARE to a separate function as the MARIA_SHARE
  can be still part of a transaction even if the table has closed.
- Changed Aria checkpoint to use the new MARIA_SHARE free function. This
  fixes a possible memory leak when using S3 tables
- Changed testing of binlog_hton to instead test for HTON_NO_ROLLBACK
- Removed checking of has_transaction_manager() in handler.cc as we can
  assume that as the transaction was started by the engine, it does
  support transactions.
- Added new class 'start_new_trans' that can be used to start indepdendent
  sub transactions, for example while reading mysql.proc, using help or
  status tables etc.
- open_system_tables...() and open_proc_table_for_Read() doesn't anymore
  take a Open_tables_backup list. This is now handled by 'start_new_trans'.
- Split thd::has_transactions() to thd::has_transactions() and
  thd::has_transactions_and_rollback()
- Added handlerton code to free cached transactions objects.
  Needed by InnoDB.

squash! 2ed35999f2a2d84f1c786a21ade5db716b6f1bbc
2020-05-23 12:29:10 +03:00
Monty
d1d472646d Change THD->transaction to a pointer to enable multiple transactions
All changes (except one) is of type
thd->transaction.  -> thd->transaction->

thd->transaction points by default to 'thd->default_transaction'
This allows us to 'easily' have multiple active transactions for a
THD object, like when reading data from the mysql.proc table
2020-05-23 12:29:10 +03:00
Sergei Golubchik
67aaf51cf9 cleanup: ha_external_unlock() helper
as mentioned in f9f33b85be and generally to make it
easier to talk about
2020-05-05 19:41:12 +02:00
Kentoku SHIBA
181f17c3cd MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2020-04-17 11:52:42 +09:00
Kentoku SHIBA
68ceb4b460 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2020-04-17 11:51:14 +09:00
Kentoku SHIBA
b4dd996dc1 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 10:32:57 +09:00
Kentoku SHIBA
00db9c6b40 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 10:31:17 +09:00
Kentoku SHIBA
619a2ccd67 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 01:49:46 +09:00
Monty
f36ca142f7 Added page_range to records_in_range() to improve range statistics
Prototype change:
-  virtual ha_rows records_in_range(uint inx, key_range *min_key,
-                                   key_range *max_key)
+  virtual ha_rows records_in_range(uint inx, const key_range *min_key,
+                                   const key_range *max_key,
+                                   page_range *res)

The handler can ignore the page_range parameter. In the case the handler
updates the parameter, the optimizer can deduce the following:
- If previous range's last key is on the same block as next range's first
  key
- If the current key range is in one block
- We can also assume that the first and last block read are cached!
  This can be used for a better calculation of IO seeks when we
  estimate the cost of a range index scan.

The parameter is fully implemented for MyISAM, Aria and InnoDB.
A separate patch will update handler::multi_range_read_info_const() to
take the benefits of this change and also remove the double
records_in_range() calls that are not anymore needed.
2020-03-27 03:54:45 +02:00
Marko Mäkelä
5f5c63e0fe Merge 10.4 into 10.5 2020-03-24 09:54:08 +02:00
Marko Mäkelä
efc97eff31 Fix clang -Wsometimes-uninitialized 2020-03-24 09:35:59 +02:00
Monty
305cffebab merge 10.4 to 10.5 2020-03-18 12:00:38 +02:00
Monty
1242eb3d32 Removed double records_in_range calls from multi_range_read_info_const
This was to remove a performance regression between 10.3 and 10.4
In 10.5 we will have a better implementation of records_in_range
that will enable us to get more statistics.
This change was not done in 10.4 because the 10.5 will be part of
a larger change that is not suitable for the GA 10.4 version

Other things:
- Changed default handler block_size to 8192 to fix things statistics
  for engines that doesn't set the block size.
- Fixed a bug in spider when using multiple part const ranges
  (Patch from Kentoku)
2020-03-17 02:16:48 +02:00
Alexander Barkov
f1e13fdc8d MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
Aleksey Midenkov
8ed646f071 Merge 10.4 into 10.5 2019-12-02 13:35:54 +03:00
Kentoku
e066723a41 MDEV-18973 CLIENT_FOUND_ROWS wrong in spider
Get count from last_used_con->info
Contributed by willhan at Tencent Games
2019-11-29 23:23:57 +09:00
Alexander Barkov
c1599821a5 Merge remote-tracking branch 'origin/10.4' into 10.5 2019-08-13 23:49:10 +04:00
Marko Mäkelä
624dd71b94 Merge 10.4 into 10.5 2019-08-13 18:57:00 +03:00
Alexander Barkov
95cdc1ca5f Merge commit '43882e764d6867c6855b1ff057758a3f08b25c55' into 10.4 2019-08-13 11:42:31 +04:00
Alexander Barkov
e978efd96b MDEV-20273 Add class Item_sum_min_max 2019-08-07 14:13:44 +04:00
Kentoku SHIBA
ddce859076
MDEV-18737 Spider "Out of memory" on armv7hl (#1363)
This is an issue of memory alignment of variable argument when calling my_multi_malloc().
The fix is strictly casting allocating size to "uint".
2019-07-25 22:52:45 +09:00
Vladislav Vaintroub
bd917e0811 Fix clang-cl warnings 2019-07-04 10:27:10 +02:00
Kentoku SHIBA
76200870ea
MDEV-17204 Mariadb 10.3.9 Spider DB SQL Alias no execute (#1348) 2019-06-29 03:42:35 +09:00
Kentoku SHIBA
d4bdf1c0b6
MDEV-16508 spider: sql_mode not maintained between spider node and data nodes (#1347)
Add the following parameter.

- spider_sync_sql_mode
  Local sql_mode synchronous existence to remote server.
   0 : It doesn't synchronize.
   1 : It synchronizes.
  The default value is 1
2019-06-29 03:35:48 +09:00
Marko Mäkelä
984d7100cd Merge 10.4 into 10.5 2019-06-13 18:36:09 +03:00
Kentoku SHIBA
be5c432a42
MDEV-16249 CHECKSUM TABLE for a spider table is not parallel and saves all data in memory in the spider head by default (#1328)
add checksum_null for setting null value of checksum
2019-06-11 00:25:08 +09:00
Alexander Barkov
9f23f8e598 MDEV-19599 Change db_name, table_name to LEX_CSTRING in Item_ident and Send_field 2019-05-26 09:49:13 +04:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Kentoku SHIBA
0cbc930616
MDEV-18992 Crash when using 'insert into on duplicate update'if session charset different from table charset (#1290) 2019-04-30 21:39:08 +09:00
Kentoku SHIBA
82d0698962
MDEV-18995 Some bugs in direct join (#1285)
Add const_table support to the direct join feature and condition pushdown feature
2019-04-25 23:28:04 +09:00