Commit graph

186937 commits

Author SHA1 Message Date
Sergey Vojtovich
609a0d3db3 Thread safe statistics loading
Previously multiple threads were allowed to load statistics concurrently.
There were no known problems caused by this. But given amount of data
races in this code, it'd happen sooner or later.

To avoid scalability bottleneck, statistics loading is protected by
per-TABLE_SHARE atomic variable.

Whenever statistics were loaded by preceding statement (hot-path), a
scalable load-acquire check is performed.

Whenever statistics have to be loaded anew, mutual exclusion for loaders
is established by atomic variable. If statistics are being loaded
concurrently, statement waits until load is completed.

TABLE_STATISTICS_CB::stats_can_be_read and
TABLE_STATISTICS_CB::stats_is_read are replaced with a tri state atomic
variable.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected
2020-05-29 21:53:54 +04:00
Sergey Vojtovich
1055a7f4fc Simplified away statistics_for_tables_is_needed()
Removed redundant loops, integrated logics into the caller instead.
Unified condition in read_statistics_for_tables(), less
"table_share != NULL" checks, no more potential "table_share == NULL"
dereferencing.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected
2020-05-29 21:53:54 +04:00
Kentoku SHIBA
38ea795bb6 Add a counter to avoid multiple initialization of Groonga mecab tokenizer 2020-05-29 21:48:47 +09:00
Kentoku SHIBA
6e6a4227c0 Add grn_db_fin_mecab_tokenizer to finalyze mecab tokenizer 2020-05-29 21:48:30 +09:00
Alexander Barkov
a2932e86b5 MDEV-22744 *SAN: sql/item_xmlfunc.cc:791:43: runtime error: downcast of address ... which does not point to an object of type 'Item_func' note: object is of type 'Item_bool' (on optimized builds)
In Item_nodeset_func_predicate::val_nodeset, args[1] is not necessarily
an Item_func descendant. It can be Item_bool.

Removing a wrong cast. It was not really needed anyway.
2020-05-29 15:31:24 +04:00
Vladislav Vaintroub
069200267d Fix cmake warning - custom command succeeds without creating own OUTPUT. 2020-05-29 12:28:34 +02:00
Vladislav Vaintroub
b00cd3e453 MDEV-22743 Windows 10 MSI installer : port in use is not determined
when checking for free port, use the same logic (IPv6 socket address
/ dual socket), like the server would.

Previous solution for testing whether port is free was trying to bind
IPv4 socket on INADDR_ANY.

This not work now on some reason, that attempt succeeds, even if there is
an existing IPv6-dual socket listening on 0.0.0.0:3306
2020-05-29 12:28:34 +02:00
Vladislav Vaintroub
ff72f36948 MSI installer : Use CAQuietExec64 on Win64 , not CAQuietExec
It works, but irritates people who look into the log and see
traces of 32bit custom action server.
2020-05-29 12:28:34 +02:00
Vladislav Vaintroub
e2d7da4982 Remove unused WiX source file 2020-05-29 12:27:55 +02:00
Aleksey Midenkov
19da9a51ae MDEV-16937 Strict SQL with system versioned tables causes issues
Respect system fields in NO_ZERO_DATE mode.

This is the subject for refactoring in MDEV-19597
2020-05-28 22:22:20 +03:00
Aleksey Midenkov
dd9773b723 MDEV-22413 Server hangs upon UPDATE on a view reading from versioned partitioned table
UPDATE gets access to history records because versioning conditions
are not set for VIEW. This leads to endless loop of inserting history
records when clustered index is rebuilt and ha_rnd_next() returns
newly inserted history record.

Return back original behavior of failing on write-locked table in
historical query.

35b679b9 assumed that SELECT_LEX::lock_type influences anything, but
actually at this point table is already locked. Original bug report
was tempesta-tech/mariadb#102
2020-05-28 22:22:19 +03:00
Anel Husakovic
a1b3bebe1f fix pre-definition for embedded server for find_user_or_anon()
Pre-definitions are allowed for non-embedded.
Failur catched with:
```
cmake ../../10.1 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_COMPILER=g++-9
-DCMAKE_C_COMPILER=gcc-9 -DWITH_EMBEDDED_SERVER=ON -DCMAKE_BUILD_TYPE=Debug
-DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,PERFSCHEMA,SPIDER,SPHINX}=N
-DMYSQL_MAINTAINER_MODE=ON -DNOT_FOR_DISTRIBUTION=ON
```
Alternative fix would be
```
--- a/sql/sql_acl.cc
+++ b/sql/sql_acl.cc
@@ -201,8 +201,10 @@ LEX_STRING current_user= { C_STRING_WITH_LEN("*current_user") };
 LEX_STRING current_role= { C_STRING_WITH_LEN("*current_role") };
 LEX_STRING current_user_and_current_role= { C_STRING_WITH_LEN("*current_user_and_current_role") };

+#ifndef EMBEDDED_LIBRARY
 class ACL_USER;
 static ACL_USER *find_user_or_anon(const char *host, const char *user, const char *ip);
+#endif
```
2020-05-28 20:18:25 +02:00
Aleksey Midenkov
3e9b96b6ff MDEV-18794 append_drop_column() small refactoring
Bogus if() logic inside the func.
2020-05-28 21:00:49 +03:00
Aleksey Midenkov
dac1280a65 MDEV-18794 Assertion `!m_innodb' failed in ha_partition::cmp_ref upon SELECT from partitioned table
System versioning assertion fix. Since DROP SYSTEM VERSIONING does not
change list of dropped keys we should handle a special case.

Caused by MDEV-19751. This fix deprecates MDEV-17091.
2020-05-28 20:55:59 +03:00
Anel Husakovic
957cb7b7ba MDEV-22312: Bad error message for SET DEFAULT ROLE when user account is not granted the role
- `SET DEFAULT ROLE xxx [FOR yyy]` should say:
  "User yyy has not been granted a role xxx" if:
    - The current user (not the user `yyy` in the FOR clause) can see the
    role xxx. It can see the role if:
      * role exists in `mysql.roles_mappings` (traverse the graph),
      * If the current user has read access on `mysql.user` table - in
    that case, it can see all roles, granted or not.
    - Otherwise it should be "Invalid role specification".

In other words, it should not be possible to use `SET DEFAULT ROLE` to discover whether a specific role exist or not.
2020-05-28 17:08:40 +02:00
Marko Mäkelä
dad7a8ee7d Merge 10.2 into 10.3 2020-05-27 17:10:39 +03:00
Sergei Golubchik
8afcc37c68 update C/C 2020-05-27 15:56:40 +02:00
Sergei Golubchik
2c9c9acbfc bintars should use bundled PCRE 2020-05-27 15:56:40 +02:00
Sergei Golubchik
9fd8f1b264 mtr: update titlebar when the test ends, not when it starts
otherwise it reaches "0 tests left" state and then waits for
a few minutes  for all workers to complete their tests.

show failures. account for retries.
2020-05-27 15:56:40 +02:00
Sergei Golubchik
e64dc07125 assert(a && b); -> assert(a); assert(b); 2020-05-27 15:56:40 +02:00
Sergei Golubchik
04726f2920 get rid of cmake warning 2020-05-27 15:56:40 +02:00
Sergei Golubchik
8cf589218f optimize performance of the build in a fresh clone
don't fetch the complete history of all submodules, it's rarely needed
2020-05-27 15:56:40 +02:00
Sergei Golubchik
cceb965a79 Revert "MDEV-12445 : Rocksdb does not shutdown worker threads and aborts in memleak check on server shutdown"
This reverts commit 6f1f911497.

because it doesn't do anything now (the server doesn't check
my_disable_leak_check) and it never did anything before
(because without `extern` it simply created a local instance of
my_disable_leak_check, did not affect server's my_disable_leak_check).
2020-05-27 15:56:40 +02:00
Sergei Golubchik
6af37ba881 fix rocksdb zstd detection
WITH_ROCKSDB_ZSTD must use the same capitalization as
in the foreach loop
2020-05-27 15:56:40 +02:00
Sergei Golubchik
ad77247866 MDEV-21958 Query having many NOT-IN clauses running forever and causing available free memory to use completely
let thd->killed to abort range optimizer
2020-05-27 15:56:40 +02:00
Sergei Golubchik
1e951155bd bugfix: use THD::main_mem_root for kill error message
cannot use the current THD::mem_root, because it can be temporarily
reassigned to something with a very different life time
(e.g. to TABLE::mem_root or range optimizer mem_root).
2020-05-27 15:56:40 +02:00
Sergei Golubchik
b01c8a6cc8 MDEV-22558 wrong error for invalid utf8 table comment 2020-05-27 15:56:40 +02:00
Sergei Golubchik
39c141b4ae don't include .git files in source packages
the existing ".git/" rule only filters out .git directories,
but as we have submodules now, we have to filter out .git files too.
2020-05-27 15:56:40 +02:00
Sergei Golubchik
a50e6c9eb1 MDEV-17153 server crash on repair table ... use_frm
data_file_length == 0 in mi_repair() is normal for REPAIR ... USE_FRM.
But in-file links (for blocks and deleted chain) must be compared with
the real file length to avoid spurious "link points outside datafile"
warnings and arbitrary block skipping.
2020-05-27 15:56:40 +02:00
Eugene Kosov
5139cfabb3 fix compilation 2020-05-27 16:37:34 +03:00
Marko Mäkelä
1b3adaab25 Fix the build with GCC 4.1.2
The build was broken on CentOS 5 and CentOS 6 as a result of
commit 18d8f06f31 introducing
some constructs that old GCC versions could not cope with.
2020-05-27 10:58:28 +03:00
Marko Mäkelä
bf1aa7569e Add an end-of-test marker to ease merges 2020-05-27 09:56:37 +03:00
Marko Mäkelä
67496281ea Fix the RelWithDebInfo build
The build was broken in the parent
commit 18d8f06f31.
2020-05-27 09:51:46 +03:00
Eugene Kosov
18d8f06f31 intrusive::list fixes
namespace intrusive: removed

split class into two: ilist<T> and sized_ilist<T> which has a size field.

ilist<T> no more NULLify pointers to bring a slignly better performance.
As a consequence, fil_space_t::is_in_unflushed_spaces and
fil_space_t::is_in_rotation_list boolean members are needed now.
2020-05-27 02:34:25 +03:00
Monty
403dacf6a9 Fixed crash in aria recovery when using bulk insert
MDEV-20578 Got error 126 when executing undo undo_key_delete
upon Aria crash recovery

The crash happens in this scenario:
- Table with unique keys and non unique keys
- Batch insert (LOAD DATA or INSERT ... SELECT) with REPLACE
- Some insert succeeds followed by duplicate key error

In the above scenario the table gets corrupted.

The bug was that we don't generate any undo entry for the
failed insert as the whole insert can be ignored by undo.
The code did however not take into account that when bulk
insert is used, we would write cached keys to the file on
failure and undo would wrongly ignore these.

Fixed by moving the writing of the cache keys after we write
the aborted-insert event to the log.
2020-05-26 20:05:17 +03:00
Andrei Elkin
0c1f97b3ab MDEV-15152 Optimistic parallel slave doesnt cope well with START SLAVE UNTIL
The immediate bug was caused by a failure to recognize a correct
position to stop the slave applier run in optimistic parallel mode.
There were the following set of issues that the analysis unveil.
1 incorrect estimate for the event binlog position passed to
  is_until_satisfied
2 wait for workers to complete by the driver thread did not account non-group events
  that could be left unprocessed and thus to mix up the last executed
  binlog group's file and position:
  the file remained old and the position related to the new rotated file
3 incorrect 'slave reached file:pos' by the parallel slave report in the error log
4 relay log UNTIL missed out the parallel slave branch in
  is_until_satisfied.

The patch addresses all of them to simplify logics of log change
notification in either the master and relay-log until case.
P.1 is addressed with passing the event into is_until_satisfied()
for proper analisis by the function.
P.2 is fixed by changes in handle_queued_pos_update().
P.4 required removing relay-log change notification by workers.
Instead the driver thread updates the notion of the current relay-log
fully itself with aid of introduced
bool Relay_log_info::until_relay_log_names_defer.

An extra print out of the requested until file:pos is arranged
with --log-warning=3.
2020-05-26 18:49:43 +03:00
Andrei Elkin
451bfcd6bb Merge remote-tracking branch 'origin/10.1' into 10.2 to skip 10.1
dbe447a789 commit.
2020-05-26 18:48:35 +03:00
Andrei Elkin
dbe447a789 MDEV-15152 Optimistic parallel slave doesnt cope well with START SLAVE UNTIL
The immediate bug was caused by a failure to recognize a correct
position to stop the slave applier run in optimistic parallel mode.
There were the following set of issues that the analysis unveil.
1 incorrect estimate for the event binlog position passed to
  is_until_satisfied
2 wait for workers to complete by the driver thread did not account non-group events
  that could be left unprocessed and thus to mix up the last executed
  binlog group's file and position:
  the file remained old and the position related to the new rotated file
3 incorrect 'slave reached file:pos' by the parallel slave report in the error log
4 relay log UNTIL missed out the parallel slave branch in
  is_until_satisfied.

The patch addresses all of them to simplify logics of log change
notification in either the master and relay-log until case.
P.1 is addressed with passing the event into is_until_satisfied()
for proper analisis by the function.
P.2 is fixed by changes in handle_queued_pos_update().
P.4 required removing relay-log change notification by workers.
Instead the driver thread updates the notion of the current relay-log
fully itself with aid of introduced
bool Relay_log_info::until_relay_log_names_defer.

An extra print out of the requested until file:pos is arranged
with --log-warning=3.
2020-05-26 18:26:50 +03:00
Aleksey Midenkov
f1f14c2092 MDEV-20015 Assertion `!in_use->is_error()' failed in TABLE::update_virtual_field
update_virtual_field() is called as part of index rebuild in
ha_myisam::repair() (MDEV-5800) which is done on bulk INSERT finish.

Assertion in update_virtual_field() was put as part of MDEV-16222
because update_virtual_field() returns in_use->is_error(). The idea:
wrongly mixed semantics of error status before update_virtual_field()
and the status returned by update_virtual_field(). The former can
falsely influence the latter.
2020-05-26 13:15:45 +03:00
Marko Mäkelä
6a26e0c719 Merge 10.1 into 10.2 2020-05-26 13:01:34 +03:00
Marko Mäkelä
adbf85fc89 Merge 5.5 into 10.1 2020-05-26 12:44:05 +03:00
Marko Mäkelä
9bbd685e8d MDEV-22513 main.processlist_notembedded Timeout in wait_until_count_sessions
Do not blindly disconnect the connection that is in WAIT_FOR
because it could happen that neither the disconnect nor the
SIGNAL would be processed before RESET would discard the signal.
2020-05-26 12:27:51 +03:00
Aleksey Midenkov
fbcfbb0e1c MDEV-19751 Wrong partitioning by KEY() after primary key dropped
Default (empty) field list in partitioning by KEY() clause is assigned
from primary key. If primary key is changed the partitioning field
list is changed as well, so repartitioning required. Not applicable to
any non-primary keys as default field list may be taken only from
primary key.
2020-05-26 11:43:43 +03:00
Thirunarayanan Balathandayuthapani
7476e8c7cd MDEV-22637 Rollback of insert fails when column reorder happens
- During column reorder table rebuild, rollback of insert fails.
Reason is that InnoDB tries to fetch the column position from
new clustered index and it exceeds default column value tuple fields.
So InnoDB should use the table column position while searching for
defaults column value.
2020-05-26 12:44:33 +05:30
Varun Gupta
76f4ae8295 MDEV-21495: Conditional jump or move depends on uninitialised value in sel_arg_range_seq_next
Initialize the parameter PARAM::max_key_part when we iterate over the ranges to get estimates
from EITS.
2020-05-26 01:57:00 +05:30
Marko Mäkelä
ecc7f305dd Merge 10.2 into 10.3 2020-05-25 19:41:58 +03:00
Marko Mäkelä
5530a93f47 MDEV-17092 use-after-poison around lock_trx_handle_wait_low
There was a race condition where the connection of the
victim of a KILL statement is disconnected while the
KILL statement is executing.

As a side effect of this fix, we will make XA PREPARE
transactions immune to KILL statements.

Starting with MariaDB 10.2, we have a pool of trx_t objects.
trx_free() would only free memory to the pool. We poison the
contents of freed objects in the pool in order to catch misuse.

trx_free(): Unpoison also trx->mysql_thd and trx->state.
This is to counter the poisoning of *trx in trx_pools->mem_free().
Unpoison only on AddressSanitizer or Valgrind, but not on MemorySanitizer.

Pool: Unpoison allocated objects only on AddressSanitizer or
Valgrind, but not on MemorySanitizer.

innobase_kill_query(): Properly protect trx, acquiring also
trx_sys_t::mutex and checking trx_t::mysql_thd and trx_t::state.
2020-05-25 18:57:14 +03:00
Marko Mäkelä
e2c749380b MDEV-22545 post-fix: Fix a test result
commit cf52dd174e failed to
adjust the result of the test main.mysqldump.
2020-05-25 17:58:12 +03:00
Varun Gupta
4a6b28c7b9 MDEV-22461: JOIN::make_aggr_tables_info(): Assertion `select_options & (1ULL << 17)' failed.
A temporary table is needed for window function computation but if only a NAMED WINDOW SPEC
is used and there is no window function, then there is no need to create a temporary
table as there is no stage to compute WINDOW FUNCTION
2020-05-25 10:51:21 +05:30
Oleksandr Byelkin
cf52dd174e MDEV-22545: my_vsnprintf behaves not as in C standard
Added parameter %T for string which should be visibly truncated.
2020-05-24 21:27:08 +02:00