Commit graph

432 commits

Author SHA1 Message Date
Sergei Golubchik
25d9d2e37f Merge branch 'bb-10.4-release' into bb-10.5-release 2021-02-15 16:43:15 +01:00
Sergei Golubchik
00a313ecf3 Merge branch 'bb-10.3-release' into bb-10.4-release
Note, the fix for "MDEV-23328 Server hang due to Galera lock conflict resolution"
was null-merged. 10.4 version of the fix is coming up separately
2021-02-12 17:44:22 +01:00
Monty
eacefbca35 MDEV-24750 Various corruptions caused by Aria subsystem...
The test case was setting aria_sort_buffer_size to MAX_ULONGLONG-1
which was not handled gracefully by my_malloc() or safemalloc().
Fixed by ensuring that the malloc functions returns 0 if the size
is too big.
I also added some protection to Aria repair:
- Limit sort_buffer_size to 16G (after that a bigger sort buffer will
  not help that much anyway)
- Limit sort_buffer_size also according to sort file size. This will
  help by not allocating less memory if someone sets the buffer size too
  high.
2021-02-03 02:22:47 +02:00
Sergei Golubchik
60ea09eae6 Merge branch '10.2' into 10.3 2021-02-01 13:49:33 +01:00
Sergei Golubchik
0d8bd7cc3a MDEV-18428 Memory: If transactional=0 is specified in CREATE TABLE, it is not possible to ALTER TABLE
* be strict in CREATE TABLE, just like in ALTER TABLE, because
  CREATE TABLE, just like ALTER TABLE, can be rolled back for any engine
* but don't auto-convert warnings into errors for engine warnings
  (handler::create) - this matches ALTER TABLE behavior
* and not when creating a default record, these errors are handled
  specially (and replaced with ER_INVALID_DEFAULT)
* always issue a Note when a non-unique key is truncated, because it's
  not a Warning that can be converted to an Error. Before this commit
  it was a Note for blobs and a Warning for all other data types.
2021-01-11 21:54:47 +01:00
Marko Mäkelä
6a1e655cb0 Merge 10.4 into 10.5 2020-12-02 18:29:49 +02:00
Monty
c537576495 Fixed maria.create test 2020-11-30 19:57:50 +02:00
Monty
6261b1f45e Fixed maria.create test 2020-11-30 19:49:06 +02:00
Marko Mäkelä
fabdad6857 Merge 10.4 into 10.5 2020-11-17 18:15:13 +02:00
Marko Mäkelä
f0c9903795 Merge 10.3 into 10.4 2020-11-17 13:56:20 +02:00
Monty
eae9311fa2 Do not run maria.repair with --embedded as memory usage is different 2020-11-16 13:23:39 +02:00
Marko Mäkelä
d7a5824899 Merge 10.4 into 10.5 2020-11-13 21:54:21 +02:00
Marko Mäkelä
972dc6ee98 Merge 10.3 into 10.4 2020-11-12 11:18:04 +02:00
Monty
10b2d5726f Fixed failing maria.create test
This comes from a wrong merge from 10.3
2020-11-09 19:36:10 +02:00
Marko Mäkelä
533a13af06 Merge 10.3 into 10.4 2020-11-03 14:49:17 +02:00
Monty
a876121d24 MDEV-23824 SIGSEGV in end_io_cache on REPAIR LOCAL TABLE for Aria table
Bugs fixed:
- prepare_for_repair() didn't close all open files if table opened failed
  because of out-of-memory
- If dd_recreate_table() failed, the data file was not properly restored
  from it's temporary name
- Aria repair initializing code didn't properly clear all used structs
  before calling error, which caused crashed in memory-free calls.
- maria_delete_table() didn't register if table open failed. This could
  calls my_error() to be called without returning 1 to the caller, which
  cased failures in my_ok()

Note when merging to 10.5:
- Remove the #if MYSQL_VERSION from sql_admin.cc
2020-11-02 17:15:36 +02:00
Oleksandr Byelkin
8e1e2856f2 Merge branch '10.4' into 10.5 2020-11-01 14:26:15 +01:00
Oleksandr Byelkin
80c951ce28 Merge branch '10.3' into 10.4 2020-10-31 21:06:49 +01:00
Monty
14d43f4fa6 MDEV-23222 SIGSEG in maria_create() because of double free
The crash happens because a double free in the case CREATE TABLE fails
because there is a conflicting tables on disk.

Fixed by ensuring that the double free can't happen.
2020-10-29 18:34:26 +02:00
Monty
14798d3cd1 MDEV-23159 Assertion `table_share->tmp_table != NO_TMP_TABLE || m_lock_type != 2'...
The problem was that opt_sum_query() was, as part of MIN/MAX optimization,
doing read operations on constant tables that where already closed

Fixed by ensuring we don't try to read from tables that are closed.
2020-10-29 16:22:30 +02:00
Sujatha
25ede13611 Merge branch '10.4' into 10.5 2020-09-29 16:59:36 +05:30
Monty
bb2c958132 MDEV-23296 Assertion `block->type == PAGECACHE_EMPTY_PAGE.. with aria_max_sort_file_size=0
When maria_drop_all_indexes() reset the key files, it didn't flush
the page cache. This confused the cache as there where old key blocks
marked with LSN in it and repair tried to overwrite these with PLAIN
blocks which is not allowed.
2020-09-25 13:07:04 +03:00
Monty
92b5a8bb65 MDEV-17665 Assertion `!share and other errors on concurrent Aria operations
Fixes also:
MDEV-22674 Server crash in compare_bin ... restore_table_state_after_repair

The bug was that the 'can_enable_index' variable in MyISAM and Aria was
not properly set and reset for bulk insert.
Because of this, insert...select was trying to recreate indexes while
another thread was using it, causing crashes in page cache.
2020-09-25 13:07:04 +03:00
Ian Gilfillan
ee61c8c0b7 Fix spelling mistake in error message 2020-08-21 23:00:56 +03:00
Oleksandr Byelkin
48b5777ebd Merge branch '10.4' into 10.5 2020-08-04 17:24:15 +02:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Oleksandr Byelkin
c32f71af7e Merge branch '10.2' into 10.3 2020-08-03 13:41:29 +02:00
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Otto Kekäläinen
ab48901d54 Fix spelling errors 2020-07-30 10:36:39 +10:00
Sergei Golubchik
77b7f793f9 MDEV-18496 Crash when Aria encryption is enabled but plugin not available
fix uninitialized struct member
2020-07-29 14:56:24 +02:00
Sergei Golubchik
88cbe2f081 MDEV-18496 Crash when Aria encryption is enabled but plugin not available
wait_while_table_is_used() should return an error if handler::extra() fails
2020-07-29 14:56:24 +02:00
Monty
d55f8a249e Disable maria.max_length when using valgrind (too slow) 2020-07-23 10:54:33 +03:00
Monty
56045ef94a Fix for crash in Aria LOCK TABLES + CREATE TRIGGER
MDEV-22829 SIGSEGV in _ma_reset_history on LOCK
2020-06-14 19:39:43 +03:00
Monty
dfb41fddf6 Make error messages from DROP TABLE and DROP TABLE IF EXISTS consistent
- IF EXISTS ends with a list of all not existing object, instead of a
  separate note for every not existing object
- Produce a "Note" for all wrongly dropped objects
  (like trying to do DROP SEQUENCE for a normal table)
- Do not write existing tables that could not be dropped to binlog

Other things:
MDEV-22820 Bogus "Unknown table" warnings produced upon attempt to drop
           parent table referenced by FK
This was caused by an older version of this commit patch and later fixed
2020-06-14 19:39:42 +03:00
Marko Mäkelä
4a0b56f604 Merge 10.4 into 10.5 2020-05-31 10:28:59 +03:00
Marko Mäkelä
6da14d7b4a Merge 10.3 into 10.4 2020-05-30 11:04:27 +03:00
Monty
df4ab26a6b SHOW TABLE STATUS now shows if an Aria table is transactional or not
This change also affects information_schema.tables

The create table option "transactional=0 | 1" is now always shown for
storage engines that supports both transactional/crash safe tables and
non transactional tables.

Before this patch the transactional=... option was only shown if the user
specified transactional=... in the CREATE TABLE or ALTER TABLE statement.
The reason for the change was to be able to make it easy to know if an Aria
table is transactional or not.
2020-05-29 22:47:37 +03:00
Marko Mäkelä
dad7a8ee7d Merge 10.2 into 10.3 2020-05-27 17:10:39 +03:00
Monty
403dacf6a9 Fixed crash in aria recovery when using bulk insert
MDEV-20578 Got error 126 when executing undo undo_key_delete
upon Aria crash recovery

The crash happens in this scenario:
- Table with unique keys and non unique keys
- Batch insert (LOAD DATA or INSERT ... SELECT) with REPLACE
- Some insert succeeds followed by duplicate key error

In the above scenario the table gets corrupted.

The bug was that we don't generate any undo entry for the
failed insert as the whole insert can be ignored by undo.
The code did however not take into account that when bulk
insert is used, we would write cached keys to the file on
failure and undo would wrongly ignore these.

Fixed by moving the writing of the cache keys after we write
the aborted-insert event to the log.
2020-05-26 20:05:17 +03:00
Monty
59f011943f MDEV-22686 # Assertion `trn' failed in ha_maria::start_stmt
Calling stmt rollback closed the transaction, which caused the above issue
2020-05-24 19:29:56 +03:00
Monty
4102f1589c Aria will now register it's transactions
MDEV-22531 Remove maria::implicit_commit()
MDEV-22607 Assertion `ha_info->ht() != binlog_hton' failed in
           MYSQL_BIN_LOG::unlog_xa_prepare

From the handler point of view, Aria now looks like a transactional
engine. One effect of this is that we don't need to call
maria::implicit_commit() anymore.

This change also forces the server to call trans_commit_stmt() after doing
any read or writes to system tables.  This work will also make it easier
to later allow users to have system tables in other engines than Aria.

To handle the case that Aria doesn't support rollback, a new
handlerton flag, HTON_NO_ROLLBACK, was added to engines that has
transactions without rollback (for the moment only binlog and Aria).

Other things
- Moved freeing of MARIA_SHARE to a separate function as the MARIA_SHARE
  can be still part of a transaction even if the table has closed.
- Changed Aria checkpoint to use the new MARIA_SHARE free function. This
  fixes a possible memory leak when using S3 tables
- Changed testing of binlog_hton to instead test for HTON_NO_ROLLBACK
- Removed checking of has_transaction_manager() in handler.cc as we can
  assume that as the transaction was started by the engine, it does
  support transactions.
- Added new class 'start_new_trans' that can be used to start indepdendent
  sub transactions, for example while reading mysql.proc, using help or
  status tables etc.
- open_system_tables...() and open_proc_table_for_Read() doesn't anymore
  take a Open_tables_backup list. This is now handled by 'start_new_trans'.
- Split thd::has_transactions() to thd::has_transactions() and
  thd::has_transactions_and_rollback()
- Added handlerton code to free cached transactions objects.
  Needed by InnoDB.

squash! 2ed35999f2a2d84f1c786a21ade5db716b6f1bbc
2020-05-23 12:29:10 +03:00
Monty
ff64152bc7 Fixed failing tests in buildbot
- Updated icp_tests.inc and result files
- Updates usage of records_in_range() protype in cassandra
2020-03-27 12:30:10 +02:00
Monty
eb483c5181 Updated optimizer costs in multi_range_read_info_const() and sql_select.cc
- multi_range_read_info_const now uses the new records_in_range interface
- Added handler::avg_io_cost()
- Don't calculate avg_io_cost() in get_sweep_read_cost if avg_io_cost is
  not 1.0.  In this case we trust the avg_io_cost() from the handler.
- Changed test_quick_select to use TIME_FOR_COMPARE instead of
  TIME_FOR_COMPARE_IDX to align this with the rest of the code.
- Fixed bug when using test_if_cheaper_ordering where we didn't use
  keyread if index was changed
- Fixed a bug where we didn't use index only read when using order-by-index
- Added keyread_time() to HEAP.
  The default keyread_time() was optimized for blocks and not suitable for
  HEAP. The effect was the HEAP prefered table scans over ranges for btree
  indexes.
- Fixed get_sweep_read_cost() for HEAP tables
- Ensure that range and ref have same cost for simple ranges
  Added a small cost (MULTI_RANGE_READ_SETUP_COST) to ranges to ensure
  we favior ref for range for simple queries.
- Fixed that matching_candidates_in_table() uses same number of records
  as the rest of the optimizer
- Added avg_io_cost() to JT_EQ_REF cost. This helps calculate the cost for
  HEAP and temporary tables better. A few tests changed because of this.
- heap::read_time() and heap::keyread_time() adjusted to not add +1.
  This was to ensure that handler::keyread_time() doesn't give
  higher cost for heap tables than for normal tables. One effect of
  this is that heap and derived tables stored in heap will prefer
  key access as this is now regarded as cheap.
- Changed cost for index read in sql_select.cc to match
  multi_range_read_info_const(). All index cost calculation is now
  done trough one function.
- 'ref' will now use quick_cost for keys if it exists. This is done
  so that for '=' ranges, 'ref' is prefered over 'range'.
- scan_time() now takes avg_io_costs() into account
- get_delayed_table_estimates() uses block_size and avg_io_cost()
- Removed default argument to test_if_order_by_key(); simplifies code
2020-03-27 03:58:32 +02:00
Monty
305cffebab merge 10.4 to 10.5 2020-03-18 12:00:38 +02:00
Monty
1242eb3d32 Removed double records_in_range calls from multi_range_read_info_const
This was to remove a performance regression between 10.3 and 10.4
In 10.5 we will have a better implementation of records_in_range
that will enable us to get more statistics.
This change was not done in 10.4 because the 10.5 will be part of
a larger change that is not suitable for the GA 10.4 version

Other things:
- Changed default handler block_size to 8192 to fix things statistics
  for engines that doesn't set the block size.
- Fixed a bug in spider when using multiple part const ranges
  (Patch from Kentoku)
2020-03-17 02:16:48 +02:00
Vicențiu Ciorbaru
45bc7574fb MDEV-18650: Options deprecated in previous versions - storage_engine
Remove usage of deprecated variable storage_engine. It was deprecated in 5.5 but
it never issued a deprecation warning. Make it issue a warning in 10.5.1.

Replaced with default_storage_engine.
2020-02-13 13:42:01 +02:00
Marko Mäkelä
a983b24407 Merge 10.4 into 10.5 2020-01-28 14:17:09 +02:00
Oleksandr Byelkin
fdb9b05cbb fix tests 2020-01-24 15:38:25 +01:00
Oleksandr Byelkin
bfc24bb2ec Merge branch '10.3' into 10.4 2020-01-24 14:50:23 +01:00
Oleksandr Byelkin
ceda5f724f Merge branch '10.2' into 10.3 2020-01-24 14:16:20 +01:00