Commit graph

189820 commits

Author SHA1 Message Date
Marko Mäkelä
170487473e After-merge fix: main.perror-win 2020-05-26 15:48:29 +03:00
Julius Goryavsky
e04999c460 Forgotten include files were added to check the necessary conditions for running the test 2020-05-26 14:01:13 +02:00
Julius Goryavsky
98a2c6b69e Merge branch '10.4-MDEV-22666-v2' of https://github.com/codership/mariadb-server into codership-10.4-MDEV-22666-v2 2020-05-26 13:54:02 +02:00
Aleksey Midenkov
f1f14c2092 MDEV-20015 Assertion `!in_use->is_error()' failed in TABLE::update_virtual_field
update_virtual_field() is called as part of index rebuild in
ha_myisam::repair() (MDEV-5800) which is done on bulk INSERT finish.

Assertion in update_virtual_field() was put as part of MDEV-16222
because update_virtual_field() returns in_use->is_error(). The idea:
wrongly mixed semantics of error status before update_virtual_field()
and the status returned by update_virtual_field(). The former can
falsely influence the latter.
2020-05-26 13:15:45 +03:00
Marko Mäkelä
6a26e0c719 Merge 10.1 into 10.2 2020-05-26 13:01:34 +03:00
Marko Mäkelä
adbf85fc89 Merge 5.5 into 10.1 2020-05-26 12:44:05 +03:00
Marko Mäkelä
9bbd685e8d MDEV-22513 main.processlist_notembedded Timeout in wait_until_count_sessions
Do not blindly disconnect the connection that is in WAIT_FOR
because it could happen that neither the disconnect nor the
SIGNAL would be processed before RESET would discard the signal.
2020-05-26 12:27:51 +03:00
Marko Mäkelä
ca38b6e427 Merge 10.3 into 10.4 2020-05-26 11:54:55 +03:00
Aleksey Midenkov
fbcfbb0e1c MDEV-19751 Wrong partitioning by KEY() after primary key dropped
Default (empty) field list in partitioning by KEY() clause is assigned
from primary key. If primary key is changed the partitioning field
list is changed as well, so repartitioning required. Not applicable to
any non-primary keys as default field list may be taken only from
primary key.
2020-05-26 11:43:43 +03:00
Thirunarayanan Balathandayuthapani
7476e8c7cd MDEV-22637 Rollback of insert fails when column reorder happens
- During column reorder table rebuild, rollback of insert fails.
Reason is that InnoDB tries to fetch the column position from
new clustered index and it exceeds default column value tuple fields.
So InnoDB should use the table column position while searching for
defaults column value.
2020-05-26 12:44:33 +05:30
Varun Gupta
76f4ae8295 MDEV-21495: Conditional jump or move depends on uninitialised value in sel_arg_range_seq_next
Initialize the parameter PARAM::max_key_part when we iterate over the ranges to get estimates
from EITS.
2020-05-26 01:57:00 +05:30
Marko Mäkelä
ecc7f305dd Merge 10.2 into 10.3 2020-05-25 19:41:58 +03:00
sjaakola
1af6e92f0b MDEV-22666 galera.MW-328A hang
The hang can happen between a lock connection issuing KILL CONNECTION for a victim,
which is in committing phase.
There happens two resource deadlockwhere  killer is holding victim's
LOCK_thd_data and requires trx mutex for the victim.
The victim, otoh, holds his own trx mutex, but requires LOCK_thd_data
in wsrep_commit_ordered(). Hence a classic two thread deadlock happens.

The fix in this commit changes innodb commit so that wsrep_commit_ordered()
is not called while holding trx mutex. With this, wsrep patch commit time mutex
locking does not violate the locking protocol of KILL command
(i.e. LOCK_thd_data -> trx mutex)

Also, a new test case has been added in galera.galera_bf_kill.test for scenario
where a client connection is killed in committting phase.
2020-05-25 19:30:23 +03:00
Marko Mäkelä
5530a93f47 MDEV-17092 use-after-poison around lock_trx_handle_wait_low
There was a race condition where the connection of the
victim of a KILL statement is disconnected while the
KILL statement is executing.

As a side effect of this fix, we will make XA PREPARE
transactions immune to KILL statements.

Starting with MariaDB 10.2, we have a pool of trx_t objects.
trx_free() would only free memory to the pool. We poison the
contents of freed objects in the pool in order to catch misuse.

trx_free(): Unpoison also trx->mysql_thd and trx->state.
This is to counter the poisoning of *trx in trx_pools->mem_free().
Unpoison only on AddressSanitizer or Valgrind, but not on MemorySanitizer.

Pool: Unpoison allocated objects only on AddressSanitizer or
Valgrind, but not on MemorySanitizer.

innobase_kill_query(): Properly protect trx, acquiring also
trx_sys_t::mutex and checking trx_t::mysql_thd and trx_t::state.
2020-05-25 18:57:14 +03:00
Marko Mäkelä
e2c749380b MDEV-22545 post-fix: Fix a test result
commit cf52dd174e failed to
adjust the result of the test main.mysqldump.
2020-05-25 17:58:12 +03:00
Vladislav Vaintroub
8569dac1ec allow thread_pool_size_basic on Windows, in generic threadpool mode 2020-05-25 14:54:11 +02:00
Vladislav Vaintroub
e6f0371556 MDEV-22696 TP_pool_generic::set_pool_size logic so that it marks each connection to change group before the next socket read. 2020-05-25 14:54:11 +02:00
Vladislav Vaintroub
9aa6042a0d MDEV-22696 Threadpool : make sure thd->event_scheduler.data does not change as long as THD is in server_threads. 2020-05-25 14:54:11 +02:00
Vladislav Vaintroub
17437eb259 Threadpool - support changing group on Windows with generic thread pool 2020-05-25 14:54:00 +02:00
Marko Mäkelä
d8ea11a33f MDEV-22669 fixup: WITH_MSAN build fix 2020-05-25 09:47:35 +03:00
Varun Gupta
4a6b28c7b9 MDEV-22461: JOIN::make_aggr_tables_info(): Assertion `select_options & (1ULL << 17)' failed.
A temporary table is needed for window function computation but if only a NAMED WINDOW SPEC
is used and there is no window function, then there is no need to create a temporary
table as there is no stage to compute WINDOW FUNCTION
2020-05-25 10:51:21 +05:30
Oleksandr Byelkin
cf52dd174e MDEV-22545: my_vsnprintf behaves not as in C standard
Added parameter %T for string which should be visibly truncated.
2020-05-24 21:27:08 +02:00
Monty
59f011943f MDEV-22686 # Assertion `trn' failed in ha_maria::start_stmt
Calling stmt rollback closed the transaction, which caused the above issue
2020-05-24 19:29:56 +03:00
Monty
6532b10040 Fixed failure in sysvars_server_embedded,32bit
Updated the corresponding rdiff file
2020-05-23 15:02:00 +03:00
Monty
d8e2fa0c49 Fixed compiler failure on windows 2020-05-23 14:58:33 +03:00
Monty
be647ff14d Fixed deadlock with LOCK TABLES and ALTER TABLE
MDEV-21398 Deadlock (server hang) or assertion failure in
Diagnostics_area::set_error_status upon ALTER under lock

This failure could only happen if one locked the same table
multiple times and then did an ALTER TABLE on the table.

Major change is to change all instances of
table->m_needs_reopen= true;
to
table->mark_table_for_reopen();

The main fix for the problem was to ensure that we mark all
instances of the table in the locked_table_list and when we
reopen the tables, we first close all tables before reopening
and locking them.

Other things:
- Don't call thd->locked_tables_list.reopen_tables if there
  are no tables marked for reopen. (performance)
2020-05-23 14:58:33 +03:00
Monty
fc8359f0ac Fixed failure in flush_read_lock.test
The failure was:
mysqltest: At line 1737: query 'reap' failed: 1397: XAER_NOTA: Unknown XID

The bug was in the test case int that it executed REAP XA COMMIT before
the connection had truly disconnected
2020-05-23 12:29:10 +03:00
Monty
c779ef26b6 Fixed error in galera_vote_rejoin_ddl
The bug was that sometimes one get check errors like:
-AUTO_INCREMENT_OFFSET  4
+AUTO_INCREMENT_OFFSET  3
2020-05-23 12:29:10 +03:00
Monty
f4ddde0698 Only apply wsrep_trx_fragment_size to InnoDB tables
MDEV-22617 Galera node crashes when trying to log to slow_log table in
streaming replication mode

Other things:
- Changed name of wsrep_after_row(two arguments) to
  wsrep_after_row_internal(one argument) to not depended on the
  function signature with unused arguments.
2020-05-23 12:29:10 +03:00
Monty
c4bf4b7aef Fixed access to undefined memory found by valgrind and MSAN
When my_vsnprintf() is patched, the code protected disabled with
'WAITING_FOR_BUGFIX_TO_VSPRINTF' should be enabled again. Also all %b
formats in this patch should be revert to %s again
2020-05-23 12:29:10 +03:00
Monty
dcc0baf540 Don't print "Trying to delete tablespace several 10x times per second"
Now this InnoDB message is printed after one second of stalled operations
and then every 10 seconds
2020-05-23 12:29:10 +03:00
Monty
4ea171ffab Fixed compiler warnings from gcc and clang 5.0.1 2020-05-23 12:29:10 +03:00
Monty
9bf479b0cf Update galera to work with independent sub transactions 2020-05-23 12:29:10 +03:00
Monty
4102f1589c Aria will now register it's transactions
MDEV-22531 Remove maria::implicit_commit()
MDEV-22607 Assertion `ha_info->ht() != binlog_hton' failed in
           MYSQL_BIN_LOG::unlog_xa_prepare

From the handler point of view, Aria now looks like a transactional
engine. One effect of this is that we don't need to call
maria::implicit_commit() anymore.

This change also forces the server to call trans_commit_stmt() after doing
any read or writes to system tables.  This work will also make it easier
to later allow users to have system tables in other engines than Aria.

To handle the case that Aria doesn't support rollback, a new
handlerton flag, HTON_NO_ROLLBACK, was added to engines that has
transactions without rollback (for the moment only binlog and Aria).

Other things
- Moved freeing of MARIA_SHARE to a separate function as the MARIA_SHARE
  can be still part of a transaction even if the table has closed.
- Changed Aria checkpoint to use the new MARIA_SHARE free function. This
  fixes a possible memory leak when using S3 tables
- Changed testing of binlog_hton to instead test for HTON_NO_ROLLBACK
- Removed checking of has_transaction_manager() in handler.cc as we can
  assume that as the transaction was started by the engine, it does
  support transactions.
- Added new class 'start_new_trans' that can be used to start indepdendent
  sub transactions, for example while reading mysql.proc, using help or
  status tables etc.
- open_system_tables...() and open_proc_table_for_Read() doesn't anymore
  take a Open_tables_backup list. This is now handled by 'start_new_trans'.
- Split thd::has_transactions() to thd::has_transactions() and
  thd::has_transactions_and_rollback()
- Added handlerton code to free cached transactions objects.
  Needed by InnoDB.

squash! 2ed35999f2a2d84f1c786a21ade5db716b6f1bbc
2020-05-23 12:29:10 +03:00
Monty
d1d472646d Change THD->transaction to a pointer to enable multiple transactions
All changes (except one) is of type
thd->transaction.  -> thd->transaction->

thd->transaction points by default to 'thd->default_transaction'
This allows us to 'easily' have multiple active transactions for a
THD object, like when reading data from the mysql.proc table
2020-05-23 12:29:10 +03:00
Monty
7ae812cf2c Fix that BACKUP STAGE BLOCK_COMMIT blocks commit to the Aria engine
MDEV-22468 BACKUP STAGE BLOCK_COMMIT should block commit in the Aria engine

This is needed to ensure that mariabackup works properly with Aria tables

This code ads new calls to ha_maria::implicit_commit(). These will be
deleted by MDEV-22531 Remove maria::implicit_commit().
2020-05-23 12:29:10 +03:00
Monty
b15615631f MDEV-18286 Assertion `pagecache->cnt_for_resize_op ...
This patch is also pushed in 10.4. It's pushed separately in 10.5 as there
are some additional cases in 10.5 to take care of.

When merging if there is conflicts, use this code, not the 10.4 code.
2020-05-23 12:29:10 +03:00
Monty
7cb160961c Changed debug label XXX in ma_pagecache.cc XXX to proper lables 2020-05-23 12:29:10 +03:00
Monty
82d2dc9027 Remove unneeded this-> usage from sql_lex.cc 2020-05-23 12:29:10 +03:00
Monty
36019df323 Proper fix of User_variables_tracker::store
- Don't use 'auto' to declare variables (lazy and makes code harder to read)
- Fixed length to use correct type
2020-05-23 12:29:10 +03:00
Monty
b1fabf6cc9 Performance improvements to test if WSREP if active 2020-05-23 12:29:10 +03:00
Monty
93281221d1 Don't include mysql-test/var* files in git
I constantly store temporary 'var' directories as var2, var3 for
comparisons and having them in citool is annyoing
2020-05-23 12:29:09 +03:00
Monty
c287dc9b04 Fixed typo in variable description 2020-05-23 12:29:09 +03:00
Monty
610bb1d2c0 Fixed bug thr_lock_info_init
This bug didn't cause any issues with current usage
2020-05-23 12:29:09 +03:00
Monty
8cf166bfb2 Update libmarias3 to fix a memory leak 2020-05-23 12:29:09 +03:00
Marko Mäkelä
51f0fa4eb3 Cleanup: Remove a startup message
The GCC __atomic_ functions were removed already in
commit 2b47f8ff03,
and starting with MariaDB Server 10.4, InnoDB relies mostly
on C++11 std::atomic.
2020-05-22 23:10:52 +03:00
Marko Mäkelä
afdd6b1da1 MDEV-22669 InnoDB lacks CRC-32C acceleration on IA-32
In mysql/mysql-server@17e497bdb7
MySQL 5.6.3 introduced innodb_checksum_algorithm=crc32 and
implemented it for AMD64 using the SSE 4.2 instructions
(incorrectly advertised as "SSE2" in a startup message).
It was not implemented on IA-32 or on Windows.

Since MariaDB 10.2.11 commit 2401d14e6b
we make use of the SSE4.2 CRC-32C instructions on Windows on both IA-32
and AMD64.

Let us be consistent and implement CRC-32C for IA-32 on all
available platforms. GCC 4.8.2 and GCC 4.8.5 complain
"error: PIC register clobbered by 'ebx' in 'asm'"
so we will only enable this code for IA-32 starting with GCC 5.

Also, we will clean up the implementation further after
commit 1312b4ebb6.

has_sse4_2(): Replaces ut_cpuid().

ut_crc32c_8(): Replaces ut_crc32_8_hw().

ut_crc32c_64(): Replaces ut_crc32_64_low_hw(), ut_crc32_64_hw().

ut_crc32_hw(): Rewrite.

ut_crc32c_8_sw(): Replaces ut_crc32_8_sw().

ut_crc32c_64_sw(): Replaces ut_crc32_64_low_sw(), ut_crc32_64_sw().

ut_crc32_sw(): Rewrite. Avoid code bloat and do not unroll the
ut_crc32c_64_sw() loop, because no benefit has been demonstrated.

ut_crc32_init(): Only invoke ut_crc32_slice8_table_init()
if no acceleration is available.
2020-05-22 22:59:59 +03:00
Marko Mäkelä
14f1453b35 MDEV-7318: Fix a test case 2020-05-22 19:56:55 +03:00
Alexander Barkov
ea7830eef4 MDEV-14221 Assertion `0' failed in Item::field_type_for_temporal_comparison
Item_null_result did not override type_handler() because of a wrong merge
of d8a9b524f2 (MDEV-14221) from 10.1.

Overriding type_handler().

Removing the old style field_type() method. It's not relevant any more.
2020-05-22 17:00:14 +04:00
Monty
736ca14323 Don't crash if creating sequence under XA
MDEV-22002 Assertion `!is_set() || (m_status == DA_OK_BULK &&
is_bulk_op())' failed upon CREATE TEMPORARY SEQUENCE under XA
2020-05-22 15:42:11 +03:00