Commit graph

188401 commits

Author SHA1 Message Date
Alexander Barkov
83e75b39b3 MDEV-21702 Add a data type for privileges 2020-02-11 08:10:26 +04:00
Aleksey Midenkov
f79f537f9f MDEV-21690 LeakSanitizer: detected memory leaks in mem_heap_create_block_func 2020-02-10 21:57:27 +03:00
Alexander Barkov
77c6382312 MDEV-21689 Add Sql_cmd for GRANT/REVOKE statements
Rewriting GRANT/REVOKE grammar to use more bison stack and use Sql_cmd_ style

1. Removing a few members from LEX:
   - uint grant, grant_to_col, which_columns
   - List<LEX_COLUMN> columns
   - bool all_privileges
2. Adding classes Grand_object_name, Lex_grant_object_name
3. Adding classes Grand_privilege, Lex_grand_privilege
4. Adding struct Lex_column_list_privilege_st, class Lex_column_list_privilege
5. Rewriting the GRANT/REVOKE grammar to use new classes and pass them through
   bison stack (rather than directly access LEX members)
6. Adding classes Sql_cmd_grant* and Sql_cmd_revoke*,
   changing GRANT/REVOKE to use LEX::m_sql_cmd.
7. Adding the "sp_handler" grammar rule and removing some duplicate grammar
   for GRANT/REVOKE for different kinds of SP objects.
8. Adding a new rule comma_separated_ident_list, reusing it in:
   - with_column_list
   - colum_list_privilege
2020-02-08 21:35:35 +04:00
Marko Mäkelä
06b0623adb Cleanup: Aligned InnoDB index page header access
ut_align_down(): Preserve the const qualifier. Use C++ casts.

ha_delete_hash_node(): Correct an assertion expression.

fil_page_get_type(): Perform an assumed-aligned read.

page_align(): Preserve the const qualifier. Assume (some) alignment.

page_get_max_trx_id(): Check the index page type.

page_header_get_field(): Perform an assumed-aligned read.

page_get_autoinc(): Perform an assumed-aligned read.

page_dir_get_nth_slot(): Perform an assumed-aligned read.
Preserve the const qualifier.
2020-02-08 14:12:59 +02:00
Marko Mäkelä
c5856b0a68 MDEV-21351: Allocate aligned memory
recv_sys_t::ALIGNMENT: The recv_sys_t::alloc() alignment
2020-02-08 11:47:42 +02:00
Marko Mäkelä
6eed99f1f3 MDEV-21248: Do not break the build on clang 2020-02-08 11:40:55 +02:00
Martin Liska
35c2778519 MDEV-21248: Prevent optimizing out buf argument in check_stack_overrun.
When using LTO, one can see optimization of stack variables that
are passed to check_stack_overrun as argument buf. That prevents
proper stack overrun detection.
2020-02-08 05:50:01 +01:00
Marko Mäkelä
0d1ca19383 One more fixup for sizeof(mtr_t) reduction
Add explicit casts when assigning ulint to m_user_space_id.
2020-02-07 13:44:13 +02:00
Marko Mäkelä
91e7b44399 mtr_t::get_log_mode(): Remove a redundant assertion
mtr_log_t and mtr_t::m_log_mode have the same range 0 to 3.
2020-02-07 13:29:08 +02:00
Marko Mäkelä
2b260f2ddd Fixup the parent commit
mtr_t::get_log_mode(): Use equivalent static_assert().

mtr_t::m_n_log_recs: Do not exceed the number of bits in uint16_t.
2020-02-07 13:15:33 +02:00
Marko Mäkelä
9a999469f7 Cleanup: Recude sizeof(mtr_t)
Use bit-fields for some mtr_t members to improve locality of reference.
Because mtr_t is never shared between threads, there are no considerations
regarding concurrent access.
2020-02-07 12:07:12 +02:00
Marko Mäkelä
8b6cfda631 Merge 10.4 into 10.5 2020-02-07 08:51:20 +02:00
Marko Mäkelä
8b97eba31b MDEV-21674 purge_sys.stop() fails to wait for purge workers to complete
Since commit 5e62b6a5e0 (MDEV-16264),
purge_sys_t::stop() no longer waited for all purge activity to stop.

This caused problems on FLUSH TABLES...FOR EXPORT because of
purge running concurrently with the buffer pool flush.
The assertion at the end of buf_flush_dirty_pages() could fail.

The, implemented by Vladislav Vaintroub, aims to eliminate race
conditions when stopping or resuming purge:

waitable_task::disable(): Wait for the task to complete, then replace
the task callback function with noop.

waitable_task::enable(): Restore the original task callback function
after disable().

purge_sys_t::stop(): Invoke purge_coordinator_task.disable().

purge_sys_t::resume(): Invoke purge_coordinator_task.enable().

purge_sys_t::running(): Add const qualifier, and clarify the comment.
The purge coordinator task will remain active as long as any purge
worker task is active.

purge_worker_callback(): Assert purge_sys.running().

srv_purge_wakeup(): Merge with the only caller purge_sys_t::resume().

purge_coordinator_task: Use static linkage.
2020-02-07 08:12:58 +02:00
Marko Mäkelä
cd3bdc09db MDEV-18582: Fix a race condition
srv_export_innodb_status(): While gathering
innodb_mem_adaptive_hash, acquire btr_search_latches[i]
in order to prevent a race condition with buffer pool resizing.
2020-02-06 14:52:11 +02:00
Marko Mäkelä
6d214415c9 MDEV-21351: Free processed recv_sys_t::blocks
Release memory as soon as redo log records are processed.

Because the memory allocation and deallocation of parsed redo log
records must be protected by recv_sys.mutex, it is better to avoid
using a std::atomic field for bookkeeping.

buf_page_t::access_time: Keep track of the recv_sys.pages record
allocations. The most significant 16 bits will count allocated
blocks (which were previously counted by buf_page_t::buf_fix_count
in the debug version), and the least significant 16 bits indicate
the number of allocated bytes in the block (which was previously
managed in buf_block_t::modify_clock), which must be a positive
number, up to innodb_page_size. The byte offset 65536 is represented
as the value 0.

recv_recover_page(): Let the caller erase the log.

recv_validate_tablespace(): Acquire recv_sys_t::mutex.
2020-02-06 09:00:19 +02:00
Oleksandr Byelkin
c1eaa385ff MDEV-21616: Server crash when using "SET STATEMENT max_statement_time=0 FOR desc xxx" lead to collapse
Main select should be pushed first.
2020-02-05 13:37:12 +01:00
Marko Mäkelä
2acc6f2d95 MDEV-21658 Error on online ADD PRIMARY KEY after instant DROP/reorder
row_log_table_get_pk_old_col(): For replacing a NULL value for a
column of the being-added primary key, look up the correct
default value, even if columns had been instantly reordered or
dropped earlier. This ought to have been broken ever since
commit 0e5a4ac253 (MDEV-15562).
2020-02-05 11:12:10 +02:00
mkaruza
d0c8316bf5
Incorrect behaviour of WSREP_SYNC_WAIT_UPTO_GTID (#1442)
Function `signal_waiters` assigned `m_committed_seqno` variable outside of
mutex lock which caused incorrect behavior of WSREP_SYNC_WAIT_UPTO_GTID.
Fixed by moving assignment inside lock. Added handling of OOM and now
error is reported.
Remove hard-coded seqno value and read seqno directly from current node state.
2020-02-05 10:02:33 +02:00
Sergey Vojtovich
daaa881cfe libpmem cmake macros
Also added support for MAP_SYNC. It allows to achieve decent performance
with DAX devices even when libpmem is unavailable.

Fixed Windows version of my_msync(): according to manual FlushViewOfFile()
may return before flush is actually completed. It is advised to issue
FlushFileBuffers() after FlushViewOfFile().
2020-02-04 23:23:50 +04:00
Marko Mäkelä
a56f78243e MDEV-21645 SIGSEGV in innobase_get_computed_value
ha_innobase::commit_inplace_alter_table(): After
ALTER_STORED_COLUMN_ORDER, ensure that the virtual column metadata
will be reloaded also when the table is not being rebuilt.
2020-02-04 16:36:58 +02:00
Sujatha
42e825dd0a MDEV-20601: Make REPLICA a synonym for SLAVE in SQL statements
Fix:
===
Add "REPLICA" as an alias for "SLAVE". All commands which use "SLAVE" keyword
can be used with new alias "REPLICA".

List of commands:

On Master:
=========
SHOW REPLICA HOSTS <--> SHOW SLAVE HOSTS
Privilege "SLAVE"  <--> "REPLICA"

On Slave:
=========
START SLAVE       <--> START REPLICA
START ALL SLAVES  <--> START ALL REPLICAS
START SLAVE UNTIL <--> START REPLICA UNTIL
STOP SLAVE        <--> STOP REPLICA
STOP ALL SLAVES   <--> STOP ALL REPLICAS
RESET SLAVE       <--> RESET REPLICA
RESET SLAVE ALL   <--> RESET REPLICA ALL
SLAVE_POS         <--> REPLICA_POS
2020-02-04 18:16:21 +05:30
Jan Lindström
46386661a2 MDEV-20625 : MariaDB asserting when enabling wsrep_on
We need to release global system variables mutex before
doing wsrep_init to avoid race with next show status and
we need to save wsrep_on value as it is changed on wsrep_init.
Added test case.
2020-02-04 11:14:21 +02:00
Julius Goryavsky
93278ee8ad MDEV-20625: MariaDB asserting when enabling wsrep=on 2020-02-04 08:56:03 +02:00
Jan Lindström
574354a6b2 MDEV-20625 : MariaDB asserting when enabling wsrep_on
When wsrep_on is changed to ON we might need to run wsrep_init
if wsrep-provider is set and wsrep is not inited.
2020-02-03 19:45:30 +02:00
Eugene Kosov
287c1db786 try to fix Win x86 build 2020-02-03 17:59:14 +08:00
Sachin
eed6d215f1 MDEV-20001 Potential dangerous regression: INSERT INTO >=100 rows fail for myisam table with HASH indexes
Problem:-

So the issue is when we do bulk insert with rows
> MI_MIN_ROWS_TO_DISABLE_INDEXES(100) , We try to disable the indexes to
speedup insert. But current logic also disables the long unique indexes.

Solution:- In ha_myisam::start_bulk_insert if we find long hash index
(HA_KEY_ALG_LONG_HASH) we will not disable the index.

This commit also refactors the mi_disable_indexes_for_rebuild function,
Since this is function is called at only one place, it is inlined into
start_bulk_insert

mi_clear_key_active is added into myisamdef.h because now it is also used
in ha_myisam.cc file.

(Same is done for Aria Storage engine)
2020-02-03 12:44:31 +05:30
Aleksey Midenkov
b615d275b8 MDEV-17798 System variable system_versioning_asof accepts wrong values (10.4) 2020-02-02 17:13:58 +03:00
Sachin Setiya
5a6023cf6f MDEV-18791 Wrong error upon creating Aria table with long index on BLOB
If we have long unique key for aria engine return too long key error, because
Aria does not support key on virtual generated column.
2020-02-02 13:53:26 +05:30
Eugene Kosov
691c691adc clean up redo log
main change: rename first redo log without file close

second change: use os_offset_t to represent offset in a file

third change: fix log texts
2020-02-01 23:58:24 +08:00
Marko Mäkelä
1b414c0313 MDEV-21256 after-merge fix: Use std::atomic
Starting with MariaDB Server 10.4, C++11 is being used.
Hence, std::atomic should be preferred to my_atomic.
2020-02-01 15:06:12 +02:00
Marko Mäkelä
4b291588bb MDEV-19845: Make my_cpu.h self-contained
Fix up commit f5c080c735
2020-02-01 14:56:05 +02:00
Marko Mäkelä
d87b725eeb MDEV-17844 recs_off_validate() fails in page_zip_write_trx_id_and_roll_ptr()
In commit 0e5a4ac253 (MDEV-15562)
we introduced was a bogus debug check failure that does not affect
the correctness of the release build.

With a fixed-length PRIMARY KEY, we do not have to recompute
the rec_get_offsets() after restarting the mini-transaction,
because the offsets of DB_TRX_ID,DB_ROLL_PTR are not going
to change.

row_undo_mod_clust(): Invoke rec_offs_make_valid() to keep the
debug check in page_zip_write_trx_id_and_roll_ptr() happy.

The scenario to reproduce this bug should be rather unlikely:
In the time frame when row_undo_mod_clust() has committed its
first mini-transaction and has not yet started the next one,
another mini-transaction must do something that causes the page
to be reorganized, split or merged.
2020-01-31 09:54:43 +02:00
Marko Mäkelä
88bcc7f21c Fixup cd2c0e013c
The variable 'dlh' was being used uninitialized if WSREP_PROVIDER
is not set.
2020-01-31 09:17:12 +02:00
Sachin
a10a94b262 Empty commit 2020-01-31 11:47:17 +05:30
mkaruza
74f7620636 MDEV-21598 Galera test galera.galera_sst_mysqldump does not take wsrep-new-cluster into account
Variable `wsrep_new_cluster` should be set to false after `wsrep_init_startup`.
Problem was that this was done before when mysqldump is used as SST method so option
wsrep-new-cluster didn't have any effect.
2020-01-30 14:53:42 +02:00
Monty
4d61f1247a Fixed compiler warnings from gcc 7.4.1
- Fixed possible error in rocksdb/rdb_datadic.cc
2020-01-29 23:23:55 +02:00
Monty
cd2c0e013c Added error output wsrep_print_version
This helps to determinate why galera library doesn't load
2020-01-29 23:14:41 +02:00
mkaruza
41bc736871 Galera GTID support
Support for galera GTID consistency thru cluster. All nodes in cluster
should have same GTID for replicated events which are originating from cluster.
Cluster originating commands need to contain sequential WSREP GTID seqno
Ignore manual setting of gtid_seq_no=X.

In master-slave scenario where master is non galera node replicated GTID is
replicated and is preserved in all nodes.

To have this - domain_id, server_id and seqnos should be same on all nodes.
Node which bootstraps the cluster, to achieve this, sends domain_id and
server_id to other nodes and this combination is used to write GTID for events
that are replicated inside cluster.

Cluster nodes that are executing non replicated events are going to have different
GTID than replicated ones, difference will be visible in domain part of gtid.

With wsrep_gtid_domain_id you can set domain_id for WSREP cluster.

Functions WSREP_LAST_WRITTEN_GTID, WSREP_LAST_SEEN_GTID and
WSREP_SYNC_WAIT_UPTO_GTID now works with "native" GTID format.

Fixed galera tests to reflect this chances.

Add variable to manually update WSREP GTID seqno in cluster

Add variable to manipulate and change WSREP GTID seqno. Next command
originating from cluster and on same thread will have set seqno and
cluster should change their internal counter to it's value.
Behavior is same as using @@gtid_seq_no for non WSREP transaction.
2020-01-29 15:06:06 +02:00
Marko Mäkelä
5defdc382b Cleanup: Remove mtr_state_t and mtr_t::m_state
mtr_t::is_active(), mtr_t::is_committed(): Make debug-only.
2020-01-29 14:28:45 +02:00
Marko Mäkelä
c69a8629f7 MDEV-21362: Do not call memcmp on null pointers
Starting with commit 373443903b
we would invoke memcmp() unconditionally, even if the length is zero.
But, a call to memcmp() is undefined if any parameter is a null pointer,
even if the length is zero.

In the following tests, a null pointer is being passed to the comparison:
vcol.vcol_keys_innodb gcol.gcol_keys_innodb main.func_group_innodb
innodb.innodb_bug53592

cmp_data(): Keep WITH_UBSAN happy and avoid potential future bugs
in optimized builds, like the one addressed by
commit fc168c3a5e (MDEV-15587).
2020-01-29 13:43:20 +02:00
Marko Mäkelä
50324ce624 MDEV-21351 Replace recv_sys.heap with list of buf_block_t
InnoDB crash recovery used a special type of mem_heap_t that
allocates backing store from the buffer pool. That incurred
a significant overhead, leading to underutilization of memory,
and limiting the maximum contiguous allocated size of a log record.

recv_sys_t::blocks: A linked list of buf_block_t that are allocated
by buf_block_alloc() for redo log records. Replaces recv_sys_t::heap.
We repurpose buf_block_t::unzip_LRU for linking the elements.

recv_sys_t::max_log_blocks: Renamed from recv_n_pool_free_frames.

recv_sys_t::max_blocks(): Accessor for max_log_blocks.

recv_sys_t::alloc(): Allocate memory from the current recv_sys_t::blocks
element, or allocate another block.  In debug builds, various free()
member functions must be invoked, because we repurpose
buf_page_t::buf_fix_count for tracking allocations.

recv_sys_t::free_corrupted_page(): Renamed from recv_recover_corrupt_page()

recv_sys_t::is_memory_exhausted(): Renamed from recv_sys_heap_check()

recv_sys_t::pages and its elements are allocated directly by the
system memory allocator.

recv_parse_log_recs(): Remove the parameter available_memory.

We rename some variables 'store_to_hash' to 'store', because
recv_sys.pages is not actually a hash table.

This is joint work with Thirunarayanan Balathandayuthapani.
2020-01-29 12:53:39 +02:00
Marko Mäkelä
a983b24407 Merge 10.4 into 10.5 2020-01-28 14:17:09 +02:00
Alexander Barkov
a915142f48 Fixing a compilation failure of Windows (introduced in MDEV-21581) 2020-01-28 15:29:05 +04:00
Oleksandr Byelkin
bc89105496 Merge branch 'bb-10.4-release' into 10.4 2020-01-28 09:42:21 +01:00
Alexander Barkov
f1e13fdc8d MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
Daniel Bartholomew
1ef8d0b45d
bump the VERSION 2020-01-27 15:12:05 -05:00
Rasmus Johansson
dd68ba74f3
Changed Travis to 10.5
Changed Travis status to show status of branch 10.5 (it was still pointing to 10.4)
2020-01-27 10:37:32 +02:00
Elena Stepanova
ba6bfc402c List of unstable tests for 10.4.12 release 2020-01-26 22:39:52 +02:00
Sergei Petrunia
ee33c4a694 Post-merge fix 2020-01-26 11:47:16 +01:00
Oleksandr Byelkin
70815ed5b9 Merge branch '10.3' into 10.4 2020-01-25 16:10:48 +01:00