Commit graph

188038 commits

Author SHA1 Message Date
Aleksey Midenkov
76063c2a13 MDEV-20494 ER_NOT_FORM_FILE or assertion upon adding partition to period table
- Fixed mysql_prepare_create_table() constraint duplicate checking;
- Refactored period constraint handling in mysql_prepare_alter_table():
  * No need to allocate new objects;
  * Keep old constraint name but exclude it from dup checking by automatic_name;
- Some minor memory leaks fixed;
- Some conceptual TODOs.
2020-04-03 23:55:48 +03:00
Sergei Petrunia
a219006636 MDEV-22014: Rowid Filtering is not displayed well in the optimizer trace
- Print the rowid filters that are available for use with each table.
- Make print_best_access_for_table() print which filter it has picked.
- Make best_access_path() print the filter for considered ref accesses.
2020-04-02 11:50:47 +03:00
Daniele Sciascia
bdcecfa22c MDEV-22021: Galera database could get inconsistent with rollback to savepoint
When binlog is disabled, WSREP will not behave correctly when
SAVEPOINT ROLLBACK is executed and we will not rollback transaction.
2020-03-31 14:18:21 +03:00
Marko Mäkelä
e2f1f88fa6 Merge 10.3 into 10.4 2020-03-30 14:50:23 +03:00
Marko Mäkelä
b092d35f13 MDEV-20590 Introduce a file format constraint to ALTER TABLE
If a table is altered using the MDEV-11369/MDEV-15562/MDEV-13134
ALGORITHM=INSTANT, it can force the table to use a non-canonical
format:

* A hidden metadata record at the start of the clustered index
is used to store each column's DEFAULT value. This makes it possible
to add new columns that have default values without rebuilding the table.

* Starting with MDEV-15562 in MariaDB Server 10.4, a BLOB in the
hidden metadata record is used to store column mappings. This makes
it possible to drop or reorder columns without rebuilding the table.
This also makes it possible to add columns to any position or drop
columns from any position in the table without rebuilding the table.

If a column is dropped without rebuilding the table, old records
will contain garbage in that column's former position, and new records
will be written with NULL values, empty strings, or dummy values.

This is generally not a problem. However, there may be cases where
users may want to avoid putting a table into this format.
For example, users may want to ensure that future UPDATE operations
after an ADD COLUMN will be performed in-place, to reduce write
amplification. (Instantly added columns are essentially always
variable-length.) Users might also want to avoid bugs similar to
MDEV-19916, or they may want to be able to export tables to
older versions of the server.

We will introduce the option innodb_instant_alter_column_allowed,
with the following values:

* never (0): Do not allow instant add/drop/reorder,
to maintain format compatibility with MariaDB 10.x and MySQL 5.x.
If the table (or partition) is not in the canonical format, then
any ALTER TABLE (even one that does not involve instant column
operations) will force a table rebuild.

* add_last (1, default in 10.3): Store a hidden metadata record that
allows columns to be appended to the table instantly (MDEV-11369).
In 10.4 or later, if the table (or partition) is not in this format,
then any ALTER TABLE (even one that does not involve column changes)
will force a table rebuild.

Starting with 10.4:

* add_drop_reorder (2, default): Like 'add_last', but allow the
metadata record to store a column map, to support instant
add/drop/reorder of columns (MDEV-15562).
2020-03-30 12:41:59 +03:00
Thirunarayanan Balathandayuthapani
f8ec3ba01b MDEV-21832 FORCE all partition to rebuild if any one of the
partition does rebuild

- In ha_innobase::commit_inplace_alter_table() assumes that all partition
should do the same kind of alter operations. During DDL, if one partition
requires table rebuild and other partition doesn't need rebuild
then all partition should be forced to rebuild.
2020-03-30 12:41:59 +03:00
Marko Mäkelä
67f2782413 Fix GCC -Wstringop-truncation 2020-03-30 12:16:07 +03:00
Marko Mäkelä
1a9b6c4c7f Merge 10.2 into 10.3 2020-03-30 11:12:56 +03:00
Varun Gupta
b11ff3d495 MDEV-22019: Sig 11 in next_breadth_first_tab | max_sort_length setting + double GROUP BY leads to crash
No need to create a temp table for aggregation if we have encountered some error.
2020-03-30 08:03:54 +05:30
Sergey Vojtovich
9eae063e79 num_worker_threads my_atomic to Atomic_counter 2020-03-27 11:41:46 +04:00
Sergey Vojtovich
e91a3ea732 shutdown_group_count my_atomic to Atomic_counter 2020-03-27 02:17:40 +04:00
Sergey Vojtovich
ed8bf7c98f next_timeout_check my_atomic to std::atomic 2020-03-27 02:08:53 +04:00
Monty
0fb84216a3 Make mysql.innodb_mysql_lock deterministic
MDEV-7861 main.innodb_mysql_lock fails sporadically in buildbot
2020-03-26 19:39:51 +02:00
Eugene Kosov
0b00c1a22f MDEV-22005 UBSAN: applying non-zero offset 2 to null pointer in my_charpos_mb()
Empty comment has a correct length.
2020-03-26 18:33:47 +03:00
Sergey Vojtovich
ba679ae52f Fix for rpl_start_stop_slave failure
One may access freed THD members after LOCK_thd_kill is released.

With original code it can happen when killing wsrep-disabled thread on a
wsrep-enabled server. With 91ab42a8 it is happening on a wsrep-disabled
server.
2020-03-26 19:19:41 +04:00
Sergei Petrunia
af4b2ae858 MDEV-21887: federatedx crashes on SELECT ... INTO query in select_handler code
Backport to 10.4:

- Don't try to push down SELECTs that have a side effect

- In case the storage engine did support pushdown of SELECT with an INTO
  clause, write the rows we've got from it into select->join->result,
  and not thd->protocol.  This way, SELECT ... INTO ... FROM
  smart_engine_table will put the result into where instructed, and
  NOT send it to the client.
2020-03-26 15:01:44 +03:00
Sergey Vojtovich
1c8de231a3 dequeued_count my_atomic to Atomic_counter
Also allocate inuse_relaylog with new rather than my_malloc(MY_ZEROFILL).
2020-03-25 23:49:38 +04:00
Vladislav Vaintroub
98fc6b923f MDEV-20388 : disable inline assembly in WolfSSL if MSAN is on
A desperate attempt to workaround MemorySanitizer deficiencies.
2020-03-25 19:45:37 +01:00
Vladislav Vaintroub
3a1075b93e MDEV-19519 mysql_install_db.exe doesn't set password_last_changed for newly created password 2020-03-25 18:18:08 +01:00
Kentoku SHIBA
497a4169df change from to for adding defaults-file in Spider tests 2020-03-25 22:46:01 +09:00
seppo
5918b17004
MDEV-21473 conflicts with async slave BF aborting (#1475)
If async slave thread (slave SQL handler), becomes a BF victim, it may occasionally happen that rollbacker thread is used to carry out the rollback instead of the async slave thread.
This can happen, if async slave thread has flagged "idle" state when BF thread tries to figure out how to kill the victim.
The issue was possible to test by using a galera cluster as slave for external master, and issuing high load of conflicting writes through async replication and directly against galera cluster nodes.
However, a deterministic mtr test for the "conflict window" has not yet been worked on.

The fix, in this patch makes sure that async slave thread state is never set to IDLE. This prevents the rollbacker thread to intervene.
The wsrep_query_state change was refactored to happen by dedicated function to make controlling the idle state change in one place.
2020-03-24 11:01:42 +02:00
Marko Mäkelä
efc97eff31 Fix clang -Wsometimes-uninitialized 2020-03-24 09:35:59 +02:00
Marko Mäkelä
8b647d6960 MDEV-22020: Fix spider/bugfix.return_found_rows_update
The test failed to specify default-character-set when
invoking the client. The compile-time default parameters of the
client could be overridden by configuration files in /etc/mysql.
Let us explicitly specify --default-character-set.
2020-03-24 09:35:36 +02:00
Igor Babaev
caf110fa52 MDEV-21883 Server crashes when joining a subselect with 32 tables and GROUP BY
This bug could cause a crash for any query that used a derived table/view/CTE
whose specification was a SELECT with a GROUP BY clause and a FROM list
containing 32 or more table references.
The problem appeared only in the cases when the splitting optimization
could be applied to such derived table/view/CTE.
2020-03-23 19:21:57 -07:00
Eugene Kosov
a7cbce06d4 unoptimized -fsanitize=undefined build on clang requires more stack space 2020-03-23 17:42:57 +03:00
Eugene Kosov
fb74de9728 MDEV-22006 runtime error: null pointer passed as argument 2, which is declared to never be null in JOIN::copy_ref_ptr_array()
Do not memcpy() a zero-length buffer
2020-03-23 14:30:18 +03:00
Thirunarayanan Balathandayuthapani
6697135c6d MDEV-21572 buf_page_get_gen() should apply buffered page initialized
redo log during recovery

- InnoDB unnecessarily reads the page even though it has fully initialized
buffered redo log records. Allow the page initialization redo log to
apply for the page in buf_page_get_gen() during recovery.
- Renamed buf_page_get_gen() to buf_page_get_low()
- Newly added buf_page_get_gen() will check for buffered redo log for
the particular page id during recovery
- Added new function buf_page_mtr_lock() which basically latches the page
for the given latch type.
- recv_recovery_create_page() is inline function which creates a page
if it has page initialization redo log records.
2020-03-23 16:41:48 +05:30
Eugene Kosov
1e6be69380 MDEV-19658 UBSAN: runtime error: load of value 2779096485, which is not a valid value for type 'enum_binlog_format'
This is an uninitialized read.

THD::THD: initialize current_stmt_binlog_format member
2020-03-23 13:46:09 +03:00
Vladislav Vaintroub
82b3f1a80f MDEV-21930 RocksDB does not compile anymore, with Visual Studio
Update submodule, change the source file list
rename CACHE_LINE_SIZE in ut0counter, so it does not conflics with
rocksdb sources, which also defines this constant now.
2020-03-23 11:25:01 +01:00
Alice Sherepa
ad6e421bd2 MDEV-21360 restore debud_dbug through a session variable instead of '-d,..' 2020-03-23 10:57:21 +01:00
Thirunarayanan Balathandayuthapani
f7599f4799 MDEV-21792 Server aborts upon attempt to create foreign key on spatial field
- mbmaxlen is always 0 for binary field. Ignore the assert of checking
field->prefix_len % field->col->mbmaxlen == 0.
2020-03-23 13:18:00 +05:30
Eugene Kosov
7e168634e9 blind fix for Windows building 2020-03-21 20:37:00 +03:00
Sergey Vojtovich
81f700015e Cleanup my_atomic.h includes
my_atomic.h is included indirectly anyways.
2020-03-21 20:11:44 +04:00
Sergey Vojtovich
6acddd5367 global_query_id: my_atomic to Atomic_counter 2020-03-21 19:58:44 +04:00
Eugene Kosov
5e9e0b8e3b MDEV-21993 asan failure in encryption.innochecksum
simplify fix

field_ref_zero: make bigger

buf_is_zeroes(): remove a loop and check in one go
2020-03-21 17:57:04 +03:00
Sergey Vojtovich
62687801ff tc_active_instances: my_atomic to std::atomic 2020-03-21 18:37:19 +04:00
Sergey Vojtovich
3b3f931570 Discovery counters: my_atomic to Atomic_counter 2020-03-21 18:37:15 +04:00
Eugene Kosov
23993c0036 MDEV-21993 asan failure in encryption.innochecksum
buf_is_zeroes(): stop assuming that argument buffer size
is always a multiply of 4096. And thus stop reading past
that buffer.
2020-03-21 17:08:52 +03:00
Sergey Vojtovich
a39d92ca57 gtid_pos_table: my_atomic to std::atomic 2020-03-21 17:36:38 +04:00
Sergey Vojtovich
4d9977e5ff default_gtid_pos_table: my_atomic to std::atomic 2020-03-21 15:55:00 +04:00
Marko Mäkelä
de9072ca19 Connect: Remove some unused variables 2020-03-21 12:52:07 +02:00
Daniele Sciascia
9394cc8914
MDEV-21675: Data inconsistency after multirow insert rollback (#1474)
* Remove dead code

* MDEV-21675 Data inconsistency after multirow insert rollback

This patch fixes data inconsistencies that happen after rollback of
multirow inserts, with binlog disabled.
For example, statements such as `INSERT INTO t1 VALUES (1,'a'),(1,'b')`
that fail with duplicate key error. In such cases the whole statement
is rolled back. However, with wsrep_emulate_binlog in effect, the
IO_CACHE would not be truncated, and the pending rows events would be
replicated to the rest of the cluster. In the above example, it would
result in row (1,'a') being replicated, whereas locally the statement
is rolled back entirely. Making the cluster inconsistent.
The patch changes the code so that prior to statement rollback,
pending rows event are removed and the stmt cache reset.
That patch also introduces MTR tests that excercise multirow insert
statements for regular, and streaming replication.
2020-03-21 09:17:28 +02:00
Marko Mäkelä
bd3c8f47cd Merge 10.3 into 10.4 2020-03-20 22:06:55 +02:00
Marko Mäkelä
fd5c36bed5 MDEV-21959: Fix a type mismatch on 64-bit systems 2020-03-20 21:41:39 +02:00
Eugene Kosov
45973ec610 InnoDB: reduce size of dtuple_t
Making a linked list of dtuple_t is needed only for inserting
records. It's better to store tuples in a non-intrusive
container to not affect all other use cases of dtuple_t

dtuple_t::tuple_list: removed, it was 2 * sizeof(void*) bytes

ins_node_t::entry_list: now it's std::vector<dtuple_t*>

ins_node_t::entry: now it's std::vector<dtuple_t*>::iterator

DBUG_EXECUTE_IF("row_ins_skip_sec": this dead code removed
2020-03-20 21:35:42 +03:00
Eugene Kosov
54b2da9535 correct comment in buf_page_is_corrupted() 2020-03-20 21:35:42 +03:00
Eugene Kosov
1f53335d37 st_::span fixes
move span.h to a proper place to make it available for the whole server

Reformat it.

Constuctors from a contigous container are fixed
to use cont.data() instead of cont.begin()

span<>::index_type is replaced with span<>::size_type
2020-03-20 21:35:42 +03:00
Eugene Kosov
884d22f288 remove fishy reinterpret_cast from buf_page_is_zeroes()
In my micro-benchmarks memcmp(4196) 3 times faster than old
implementation. Also, it's generally better to use as less
reinterpret_casts<> as possible.

buf_is_zeroes(): renamed from buf_page_is_zeroes() and
argument changed to span<> for convenience.

st_::span<T>::const_iterator: fixed

page_zip-verify_checksum(): make argument byte* instead of void*
2020-03-20 21:35:42 +03:00
Igor Babaev
2bde065525 MDEV-17177 Crash in Item_func_in::cleanup() for SELECT executed via
prepared statement

The method Item_func_in::build_clone() that builds a clone item for an
Item_func_in item first calls a generic method Item_func::build_item()
that builds the the clones for the arguments of the Item_func_in item
to be cloned, creates a copy of the Item_func_in object and attaches the
clones for the arguments to this copy. Then the method Item_func_in::build_clone()
makes the copy fully independent on the copied object in order to
guarantee a proper destruction of the clone. The fact is the copy of the
Item_func_in object is registered as any other item object and should be
destructed as any other item object.
If the method Item_func::build_item fails to build a clone of an argument
then it returns 0. In this case no copy of the Item_func_in object should
be created. Otherwise the finalizing actions for this copy would not be
performed and the copy would remain in a state that would prevent its
proper destruction.

The code of Item_func_in::build_clone() before this patch created the copy
of the Item_func_in object before cloning the argument items. If this
cloning failed the server crashed when trying to destruct the copy item.

The code of Item_row::build_clone() was changed similarly to the code of
Item_func::build_clone though this code could not cause any problems.
2020-03-20 09:36:25 -07:00
Marko Mäkelä
44298e4dea Merge 10.2 into 10.3
Also, clean up the test innodb_gis.geometry a little further.
2020-03-20 18:12:17 +02:00