Commit graph

383 commits

Author SHA1 Message Date
Kentoku SHIBA
c34deb4cd2 fix memory calculate for spider_mon_table_cache 2020-06-05 17:29:58 +09:00
Kentoku SHIBA
d3a6ed0550 fix divided lock table issue of Spider 2020-06-05 17:29:58 +09:00
Kentoku SHIBA
418f16116f use ifdef for unused attributes for Spider 2020-06-05 17:29:58 +09:00
Kentoku SHIBA
e954d9de88 MDEV-19002 Spider performance optimization with partition
Change the following function for batch call instead of each partition
- store_lock
- external_lock
- start_stmt
- extra
- cond_push
- info_push
- top_table
2020-06-05 17:29:57 +09:00
Kentoku
132d5822e2 MENT-458 MTR Big test "spider/bugfix.sql_mode_mariadb & myself" are both failing on Azure MTR pipeline
Support the dash number of MariaDB versions by Spider's tests
2020-06-01 12:04:34 +09:00
Kentoku
1d393fed71 MENT-456 MTR Big test "spider.show_system_tables" is failing on Azure MTR pipeline
Support the dash number of MariaDB versions by Spider's install sequence
2020-06-01 12:04:34 +09:00
Monty
4102f1589c Aria will now register it's transactions
MDEV-22531 Remove maria::implicit_commit()
MDEV-22607 Assertion `ha_info->ht() != binlog_hton' failed in
           MYSQL_BIN_LOG::unlog_xa_prepare

From the handler point of view, Aria now looks like a transactional
engine. One effect of this is that we don't need to call
maria::implicit_commit() anymore.

This change also forces the server to call trans_commit_stmt() after doing
any read or writes to system tables.  This work will also make it easier
to later allow users to have system tables in other engines than Aria.

To handle the case that Aria doesn't support rollback, a new
handlerton flag, HTON_NO_ROLLBACK, was added to engines that has
transactions without rollback (for the moment only binlog and Aria).

Other things
- Moved freeing of MARIA_SHARE to a separate function as the MARIA_SHARE
  can be still part of a transaction even if the table has closed.
- Changed Aria checkpoint to use the new MARIA_SHARE free function. This
  fixes a possible memory leak when using S3 tables
- Changed testing of binlog_hton to instead test for HTON_NO_ROLLBACK
- Removed checking of has_transaction_manager() in handler.cc as we can
  assume that as the transaction was started by the engine, it does
  support transactions.
- Added new class 'start_new_trans' that can be used to start indepdendent
  sub transactions, for example while reading mysql.proc, using help or
  status tables etc.
- open_system_tables...() and open_proc_table_for_Read() doesn't anymore
  take a Open_tables_backup list. This is now handled by 'start_new_trans'.
- Split thd::has_transactions() to thd::has_transactions() and
  thd::has_transactions_and_rollback()
- Added handlerton code to free cached transactions objects.
  Needed by InnoDB.

squash! 2ed35999f2a2d84f1c786a21ade5db716b6f1bbc
2020-05-23 12:29:10 +03:00
Monty
d1d472646d Change THD->transaction to a pointer to enable multiple transactions
All changes (except one) is of type
thd->transaction.  -> thd->transaction->

thd->transaction points by default to 'thd->default_transaction'
This allows us to 'easily' have multiple active transactions for a
THD object, like when reading data from the mysql.proc table
2020-05-23 12:29:10 +03:00
Sergei Golubchik
67aaf51cf9 cleanup: ha_external_unlock() helper
as mentioned in f9f33b85be and generally to make it
easier to talk about
2020-05-05 19:41:12 +02:00
Eugene Kosov
89ff4176c1 MDEV-22437 make THR_THD* variable thread_local
Now all access goes through _current_thd() and set_current_thd()
functions.

Some functions like THD::store_globals() can not fail now.
2020-05-05 18:13:31 +03:00
Kentoku SHIBA
90aad47dd9 MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
Add test cases.
2020-05-05 22:45:17 +09:00
Kentoku SHIBA
1b81e96593 MDEV-21884 MariaDB with Spider crashes on a query
Fix a test result.
2020-04-22 04:44:28 +09:00
Kentoku SHIBA
181f17c3cd MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.
When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.
2020-04-17 11:52:42 +09:00
Kentoku SHIBA
b4dd996dc1 MDEV-21884 MariaDB with Spider crashes on a query 2020-04-17 10:32:57 +09:00
Vladislav Vaintroub
93efbc390d MDEV-22214 mariadbd.exe calls function mysqld.exe, and crashes
Stop linking plugins to the server executable on Windows.
Instead, extract whole server functionality into a large DLL, called
server.dll. Link both plugins, and small server "stub" exe to it.

This eliminates plugin dependency on the name of the server executable.
It also reduces the size of the packages (since tiny mysqld.exe
and mariadbd.exe are now both linked to one big DLL)

Also, simplify the functionality of exporing all symbols from selected
static libraries. Rely on WINDOWS_EXPORT_ALL_SYMBOLS, rather than old
self-backed solution.

fix compile error

replace GetProcAddress(GetModuleHandle(NULL), "variable_name")
for server exported data with actual variable names.

Runtime loading was never required,was error prone
, since symbols could be missing at runtime, and now it actually failed,
because we do not export symbols from executable anymore, but from a shared
library

This did require a MYSQL_PLUGIN_IMPORT decoration for the plugin,
but made the code more straightforward, and avoids missing symbols at
runtime (as mentioned before).

The audit plugin is still doing some dynamic loading, as it aims to work
cross-version. Now it won't work cross-version on Windows, as it already
uses some symbols that are *not* dynamically loaded, e.g fn_format
and those symbols now exported from server.dll , when earlier they were
exported by mysqld.exe

Windows, fixes for storage engine plugin loading
after various rebranding stuff

Create server.dll containing functionality of the whole server
make mariadbd.exe/mysqld.exe a stub that is only  calling mysqld_main()

fix build
2020-04-10 19:05:26 +02:00
Marko Mäkelä
37c14690fc Merge 10.4 into 10.5 2020-03-30 19:07:25 +03:00
Marko Mäkelä
e2f1f88fa6 Merge 10.3 into 10.4 2020-03-30 14:50:23 +03:00
Marko Mäkelä
67f2782413 Fix GCC -Wstringop-truncation 2020-03-30 12:16:07 +03:00
Monty
eb483c5181 Updated optimizer costs in multi_range_read_info_const() and sql_select.cc
- multi_range_read_info_const now uses the new records_in_range interface
- Added handler::avg_io_cost()
- Don't calculate avg_io_cost() in get_sweep_read_cost if avg_io_cost is
  not 1.0.  In this case we trust the avg_io_cost() from the handler.
- Changed test_quick_select to use TIME_FOR_COMPARE instead of
  TIME_FOR_COMPARE_IDX to align this with the rest of the code.
- Fixed bug when using test_if_cheaper_ordering where we didn't use
  keyread if index was changed
- Fixed a bug where we didn't use index only read when using order-by-index
- Added keyread_time() to HEAP.
  The default keyread_time() was optimized for blocks and not suitable for
  HEAP. The effect was the HEAP prefered table scans over ranges for btree
  indexes.
- Fixed get_sweep_read_cost() for HEAP tables
- Ensure that range and ref have same cost for simple ranges
  Added a small cost (MULTI_RANGE_READ_SETUP_COST) to ranges to ensure
  we favior ref for range for simple queries.
- Fixed that matching_candidates_in_table() uses same number of records
  as the rest of the optimizer
- Added avg_io_cost() to JT_EQ_REF cost. This helps calculate the cost for
  HEAP and temporary tables better. A few tests changed because of this.
- heap::read_time() and heap::keyread_time() adjusted to not add +1.
  This was to ensure that handler::keyread_time() doesn't give
  higher cost for heap tables than for normal tables. One effect of
  this is that heap and derived tables stored in heap will prefer
  key access as this is now regarded as cheap.
- Changed cost for index read in sql_select.cc to match
  multi_range_read_info_const(). All index cost calculation is now
  done trough one function.
- 'ref' will now use quick_cost for keys if it exists. This is done
  so that for '=' ranges, 'ref' is prefered over 'range'.
- scan_time() now takes avg_io_costs() into account
- get_delayed_table_estimates() uses block_size and avg_io_cost()
- Removed default argument to test_if_order_by_key(); simplifies code
2020-03-27 03:58:32 +02:00
Monty
f36ca142f7 Added page_range to records_in_range() to improve range statistics
Prototype change:
-  virtual ha_rows records_in_range(uint inx, key_range *min_key,
-                                   key_range *max_key)
+  virtual ha_rows records_in_range(uint inx, const key_range *min_key,
+                                   const key_range *max_key,
+                                   page_range *res)

The handler can ignore the page_range parameter. In the case the handler
updates the parameter, the optimizer can deduce the following:
- If previous range's last key is on the same block as next range's first
  key
- If the current key range is in one block
- We can also assume that the first and last block read are cached!
  This can be used for a better calculation of IO seeks when we
  estimate the cost of a range index scan.

The parameter is fully implemented for MyISAM, Aria and InnoDB.
A separate patch will update handler::multi_range_read_info_const() to
take the benefits of this change and also remove the double
records_in_range() calls that are not anymore needed.
2020-03-27 03:54:45 +02:00
Kentoku SHIBA
497a4169df change from to for adding defaults-file in Spider tests 2020-03-25 22:46:01 +09:00
Kentoku SHIBA
12d59fabe2 change from to for adding defaults-file in Spider tests 2020-03-25 22:40:14 +09:00
Marko Mäkelä
5f5c63e0fe Merge 10.4 into 10.5 2020-03-24 09:54:08 +02:00
Marko Mäkelä
efc97eff31 Fix clang -Wsometimes-uninitialized 2020-03-24 09:35:59 +02:00
Marko Mäkelä
8b647d6960 MDEV-22020: Fix spider/bugfix.return_found_rows_update
The test failed to specify default-character-set when
invoking the client. The compile-time default parameters of the
client could be overridden by configuration files in /etc/mysql.
Let us explicitly specify --default-character-set.
2020-03-24 09:35:36 +02:00
Marko Mäkelä
2559075bda Correct a result 2020-03-21 12:36:55 +02:00
Monty
305cffebab merge 10.4 to 10.5 2020-03-18 12:00:38 +02:00
Monty
1242eb3d32 Removed double records_in_range calls from multi_range_read_info_const
This was to remove a performance regression between 10.3 and 10.4
In 10.5 we will have a better implementation of records_in_range
that will enable us to get more statistics.
This change was not done in 10.4 because the 10.5 will be part of
a larger change that is not suitable for the GA 10.4 version

Other things:
- Changed default handler block_size to 8192 to fix things statistics
  for engines that doesn't set the block size.
- Fixed a bug in spider when using multiple part const ranges
  (Patch from Kentoku)
2020-03-17 02:16:48 +02:00
Kentoku SHIBA
5929e222e4 fix evaluating bitmap issue in spider 2020-03-16 08:39:49 +09:00
Alexander Barkov
a1e330de5a MDEV-21743 Split up SUPER privilege to smaller privileges 2020-03-10 23:49:47 +04:00
Sergei Golubchik
cbede21d0d cleanup: pass trxid by value 2020-03-10 19:24:23 +01:00
Sergei Golubchik
c1c5222cae cleanup: PSI key is *always* the first argument 2020-03-10 19:24:23 +01:00
Sergei Golubchik
81cffda2e6 perfschema transaction instrumentation related changes 2020-03-10 19:24:23 +01:00
Sergei Golubchik
05779bc6f1 perfschema mdl related instrumentation changes 2020-03-10 19:24:22 +01:00
Sergei Golubchik
7c58e97bf6 perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00
Alexander Barkov
f1e13fdc8d MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
Kentoku
f5b76d8c40 fix compiler warnings 2019-12-06 05:10:01 +09:00
Kentoku
1b040ce570 fix compiler warnings 2019-12-06 04:59:22 +09:00
Oleksandr Byelkin
008ee867a4 Merge branch '10.2' into 10.3 2019-12-04 17:46:28 +01:00
Jan Lindström
9d9a2253c6 Merge remote-tracking branch 10.2 into 10.3
Conflicts:
	mysql-test/suite/galera/t/galera_binlog_event_max_size_max-master.opt
	mysql-test/suite/innodb/r/innodb-mdev-7513.result
	mysql-test/suite/innodb/t/innodb-mdev-7513.test
	mysql-test/suite/wsrep/disabled.def
	storage/innobase/ibuf/ibuf0ibuf.cc
2019-12-02 14:35:10 +02:00
Aleksey Midenkov
8ed646f071 Merge 10.4 into 10.5 2019-12-02 13:35:54 +03:00
Kentoku
e066723a41 MDEV-18973 CLIENT_FOUND_ROWS wrong in spider
Get count from last_used_con->info
Contributed by willhan at Tencent Games
2019-11-29 23:23:57 +09:00
willhan
3551cd32a8 MDEV-17508 Fix bug for spider when using "not like"
fix bug for spider where using "not like" (#890)

test case:
t1 is a spider engine table;
  CREATE TABLE `t1` (
  `id` int(11) NOT NULL DEFAULT '0',
  `name` char(64) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=SPIDER

query: "select * from t1 where name  not like 'x%' " would dispatch "select xxx name name like 'x%' " to remote mysqld, is wrong
2019-11-25 15:40:01 +01:00
Marko Mäkelä
c99470b366 Merge 10.4 into 10.5 2019-11-13 20:38:14 +02:00
Kentoku SHIBA
83a0eaec08
MDEV-18987 bug in "load data local infile xxx replace into " (#1408)
It's just added regression tests.
2019-11-13 00:32:27 +09:00
Marko Mäkelä
d04f2de80a Merge 10.4 into 10.5 2019-10-11 08:41:36 +03:00
Marko Mäkelä
c11e5cdd12 Merge 10.3 into 10.4 2019-10-10 11:19:25 +03:00
Oleksandr Byelkin
896b869685 MDEV-19238 Mariadb spider - crashes on where null
(fix and explanation came with MDEV-20753 (duplicate of this bug))
2019-10-09 08:55:00 +02:00
Marko Mäkelä
1333da90b5 Merge 10.4 into 10.5 2019-09-24 10:07:56 +03:00
Marko Mäkelä
5a92ccbaea Merge 10.3 into 10.4
Disable MDEV-20576 assertions until MDEV-20595 has been fixed.
2019-09-23 17:35:29 +03:00