Commit graph

189983 commits

Author SHA1 Message Date
Sergey Vojtovich
c279878493 Thread safe histograms loading
Previously multiple threads were allowed to load histograms concurrently.
There were no known problems caused by this. But given amount of data
races in this code, it'd happen sooner or later.

To avoid scalability bottleneck, histograms loading is protected by
per-TABLE_SHARE atomic variable.

Whenever histograms were loaded by preceding statement (hot-path), a
scalable load-acquire check is performed.

Whenever histograms have to be loaded anew, mutual exclusion for loaders
is established by atomic variable. If histograms are being loaded
concurrently, statement waits until load is completed.

- Table_statistics::total_hist_size moved to TABLE_STATISTICS_CB: only
  meaningful within TABLE_SHARE (not used for collected stats).
- TABLE_STATISTICS_CB::histograms_can_be_read and
  TABLE_STATISTICS_CB::histograms_are_read are replaced with a tri state
  atomic variable.
- Simplified away alloc_histograms_for_table_share().

Note: there's still likely a data race if a thread attempts accessing
histograms data after it failed to load it (because of concurrent load).
It was there previously and goes out of the scope of this effort. One way
of fixing it could be reviving TABLE::histograms_are_read and adding
appropriate checks whenever it is needed.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected
2020-05-29 21:53:54 +04:00
Sergey Vojtovich
609a0d3db3 Thread safe statistics loading
Previously multiple threads were allowed to load statistics concurrently.
There were no known problems caused by this. But given amount of data
races in this code, it'd happen sooner or later.

To avoid scalability bottleneck, statistics loading is protected by
per-TABLE_SHARE atomic variable.

Whenever statistics were loaded by preceding statement (hot-path), a
scalable load-acquire check is performed.

Whenever statistics have to be loaded anew, mutual exclusion for loaders
is established by atomic variable. If statistics are being loaded
concurrently, statement waits until load is completed.

TABLE_STATISTICS_CB::stats_can_be_read and
TABLE_STATISTICS_CB::stats_is_read are replaced with a tri state atomic
variable.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected
2020-05-29 21:53:54 +04:00
Sergey Vojtovich
1055a7f4fc Simplified away statistics_for_tables_is_needed()
Removed redundant loops, integrated logics into the caller instead.
Unified condition in read_statistics_for_tables(), less
"table_share != NULL" checks, no more potential "table_share == NULL"
dereferencing.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected
2020-05-29 21:53:54 +04:00
Marko Mäkelä
39dc461662 MDEV-22751 Uninitialized tbl_len in dict_acquire_mdl_shared()
A crash was observed where dict_acquire_mdl_shared<trylock=false>
would invoke memcpy() with an apparently uninitialized tbl_len.

dict_table_t::parse_name(): Remove an unnecessary tbl_len--
operation. (This should be mostly non-functional cleanup.)

dict_acquire_mdl_shared(): If the second dict_table_t::parse_name()
returns false, terminate the loop just like we would do on the
first invocation.
2020-05-29 17:04:53 +03:00
Oleksandr Byelkin
58f3f692b9 MDEV-22746: Assertion `(&(&pagecache->cache_lock)->m_mutex)->count > 0 && pthread_equal(pthread_self(), (&(&pagecache->cache_lock)->m_mutex)->thread)' failed in dec_counter_for_resize_op
Removed second attempt to decrease counter
2020-05-29 15:46:44 +02:00
Aleksey Midenkov
4783494a5e MDEV-22283 Server crashes in key_copy or unexpected error 156
(The table already existed in the storage engine)

Wrong algorithm of closing partitions on error doesn't close last
partition.
2020-05-29 16:19:15 +03:00
Kentoku SHIBA
38ea795bb6 Add a counter to avoid multiple initialization of Groonga mecab tokenizer 2020-05-29 21:48:47 +09:00
Kentoku SHIBA
6e6a4227c0 Add grn_db_fin_mecab_tokenizer to finalyze mecab tokenizer 2020-05-29 21:48:30 +09:00
Alexander Barkov
a2932e86b5 MDEV-22744 *SAN: sql/item_xmlfunc.cc:791:43: runtime error: downcast of address ... which does not point to an object of type 'Item_func' note: object is of type 'Item_bool' (on optimized builds)
In Item_nodeset_func_predicate::val_nodeset, args[1] is not necessarily
an Item_func descendant. It can be Item_bool.

Removing a wrong cast. It was not really needed anyway.
2020-05-29 15:31:24 +04:00
Julius Goryavsky
d74e3a56e7 Merge branch 'codership-10.4-MDEV-22666-v2' into 10.4 2020-05-29 13:23:37 +02:00
Vladislav Vaintroub
32dd58e04b Removed function declaration of a non-existing function 2020-05-29 13:05:35 +02:00
Vladislav Vaintroub
213265130e Remove some trailing whitespaces. 2020-05-29 13:05:35 +02:00
Marko Mäkelä
5bf9e0f875 MDEV-22206 Assertion "heap_no == ULINT_UNDEFINED" in trx0i_s.cc
commit d09aec7a15 (MDEV-19940)
caused a regression. We made wait_lock_get_heap_no() return
uint16_t instead of ulint, and we mostly replaced the previous
magic value ULINT_UNDEFINED with 0. But, we failed to adjust
some assertions. Furthermore, 0 is a valid although rare value
for record locks. (Record locks can be temporarily stored on
page infimum in some operations that involve multiple leaf pages.)

Let us use 0xFFFF as the magic value. Valid heap numbers
are limited to less than 9362 = innodb_page_size/(5+1+1)
when using a minimal 1-byte PRIMARY KEY and a
secondary index on a NULL or '' column.
2020-05-29 13:41:11 +03:00
Vladislav Vaintroub
069200267d Fix cmake warning - custom command succeeds without creating own OUTPUT. 2020-05-29 12:28:34 +02:00
Vladislav Vaintroub
b00cd3e453 MDEV-22743 Windows 10 MSI installer : port in use is not determined
when checking for free port, use the same logic (IPv6 socket address
/ dual socket), like the server would.

Previous solution for testing whether port is free was trying to bind
IPv4 socket on INADDR_ANY.

This not work now on some reason, that attempt succeeds, even if there is
an existing IPv6-dual socket listening on 0.0.0.0:3306
2020-05-29 12:28:34 +02:00
Vladislav Vaintroub
ff72f36948 MSI installer : Use CAQuietExec64 on Win64 , not CAQuietExec
It works, but irritates people who look into the log and see
traces of 32bit custom action server.
2020-05-29 12:28:34 +02:00
Vladislav Vaintroub
e2d7da4982 Remove unused WiX source file 2020-05-29 12:27:55 +02:00
Marko Mäkelä
2fbf751407 MDEV-22456 after-merge fix: Avoid functional change to rw_lock_s_unlock()
In the merge 9e6e43551f we replaced
direct use of std::atomic with a wrapper class, so that
dict_index_t::lock will support the default assignment operator.

As part of that change, one occurrence of std::memory_order_release
was accidentally replaced with std::memory_order_relaxed.

Thanks to Sergey Vojtovich for noticing this.
2020-05-29 12:11:37 +03:00
Aleksey Midenkov
f76a1df003 MDEV-21127 Assertion failed in key_text::key_text
Wrong condition for corner values.
2020-05-29 11:57:59 +03:00
Aleksey Midenkov
57f7b4866f MDEV-16937 Strict SQL with system versioned tables causes issues (10.4)
Respect system fields in NO_ZERO_DATE mode.

This is the subject for refactoring in MDEV-19597

Conflict resolution from 7d5223310789f967106d86ce193ef31b315ecff0
2020-05-29 11:45:19 +03:00
Alexander Barkov
cdc2508ed1 MDEV-22625 SIGSEGV in intern_find_sys_var (optimized builds)
The constructor of Lex_ident_sys returns LEX_CSTRING(NULL,0) if character set
conversion goes wrong, and raises the "wrong character string" error in
the diagnostics area.

The code in sql_yacc.yy did not check Lex_ident_sys::ptr against NULL,
so the execution entered functions that did not expect NULL (and crashed).

Fixing the code to do MYSQL_YYABORT if Lex_ident_sys::ptr is NULL
after constructing.
2020-05-29 11:15:28 +04:00
Aleksey Midenkov
19da9a51ae MDEV-16937 Strict SQL with system versioned tables causes issues
Respect system fields in NO_ZERO_DATE mode.

This is the subject for refactoring in MDEV-19597
2020-05-28 22:22:20 +03:00
Aleksey Midenkov
dd9773b723 MDEV-22413 Server hangs upon UPDATE on a view reading from versioned partitioned table
UPDATE gets access to history records because versioning conditions
are not set for VIEW. This leads to endless loop of inserting history
records when clustered index is rebuilt and ha_rnd_next() returns
newly inserted history record.

Return back original behavior of failing on write-locked table in
historical query.

35b679b9 assumed that SELECT_LEX::lock_type influences anything, but
actually at this point table is already locked. Original bug report
was tempesta-tech/mariadb#102
2020-05-28 22:22:19 +03:00
Anel Husakovic
a1b3bebe1f fix pre-definition for embedded server for find_user_or_anon()
Pre-definitions are allowed for non-embedded.
Failur catched with:
```
cmake ../../10.1 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_COMPILER=g++-9
-DCMAKE_C_COMPILER=gcc-9 -DWITH_EMBEDDED_SERVER=ON -DCMAKE_BUILD_TYPE=Debug
-DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,PERFSCHEMA,SPIDER,SPHINX}=N
-DMYSQL_MAINTAINER_MODE=ON -DNOT_FOR_DISTRIBUTION=ON
```
Alternative fix would be
```
--- a/sql/sql_acl.cc
+++ b/sql/sql_acl.cc
@@ -201,8 +201,10 @@ LEX_STRING current_user= { C_STRING_WITH_LEN("*current_user") };
 LEX_STRING current_role= { C_STRING_WITH_LEN("*current_role") };
 LEX_STRING current_user_and_current_role= { C_STRING_WITH_LEN("*current_user_and_current_role") };

+#ifndef EMBEDDED_LIBRARY
 class ACL_USER;
 static ACL_USER *find_user_or_anon(const char *host, const char *user, const char *ip);
+#endif
```
2020-05-28 20:18:25 +02:00
Aleksey Midenkov
3e9b96b6ff MDEV-18794 append_drop_column() small refactoring
Bogus if() logic inside the func.
2020-05-28 21:00:49 +03:00
Aleksey Midenkov
dac1280a65 MDEV-18794 Assertion `!m_innodb' failed in ha_partition::cmp_ref upon SELECT from partitioned table
System versioning assertion fix. Since DROP SYSTEM VERSIONING does not
change list of dropped keys we should handle a special case.

Caused by MDEV-19751. This fix deprecates MDEV-17091.
2020-05-28 20:55:59 +03:00
Anel Husakovic
957cb7b7ba MDEV-22312: Bad error message for SET DEFAULT ROLE when user account is not granted the role
- `SET DEFAULT ROLE xxx [FOR yyy]` should say:
  "User yyy has not been granted a role xxx" if:
    - The current user (not the user `yyy` in the FOR clause) can see the
    role xxx. It can see the role if:
      * role exists in `mysql.roles_mappings` (traverse the graph),
      * If the current user has read access on `mysql.user` table - in
    that case, it can see all roles, granted or not.
    - Otherwise it should be "Invalid role specification".

In other words, it should not be possible to use `SET DEFAULT ROLE` to discover whether a specific role exist or not.
2020-05-28 17:08:40 +02:00
Monty
278facee7c Added test case for query that was crashing in 10.4.13 2020-05-28 16:56:37 +03:00
Marko Mäkelä
c2a929185c MDEV-17491 post-fix: GCC 7 -Wclass-memaccess
commit 3a37644a29 added a non-POD
member buf_page_info_t::id, and thus GCC 7 or later would complain
about a memset() call. Let my_malloc fill the memory for us.
2020-05-27 18:56:49 +03:00
Sergey Vojtovich
d72594d45d Fixed main.implicit_commit ASAN failure
InnoDB mutex monitor is accessing mutexes of poisoned (cached) trx
objects. Unpoison ReadView::m_mutex similarly to trx_t::mutex.

This is regression after MDEV-22593.
2020-05-27 19:43:20 +04:00
Marko Mäkelä
dad7a8ee7d Merge 10.2 into 10.3 2020-05-27 17:10:39 +03:00
Sergei Golubchik
8afcc37c68 update C/C 2020-05-27 15:56:40 +02:00
Sergei Golubchik
2c9c9acbfc bintars should use bundled PCRE 2020-05-27 15:56:40 +02:00
Sergei Golubchik
9fd8f1b264 mtr: update titlebar when the test ends, not when it starts
otherwise it reaches "0 tests left" state and then waits for
a few minutes  for all workers to complete their tests.

show failures. account for retries.
2020-05-27 15:56:40 +02:00
Sergei Golubchik
e64dc07125 assert(a && b); -> assert(a); assert(b); 2020-05-27 15:56:40 +02:00
Sergei Golubchik
04726f2920 get rid of cmake warning 2020-05-27 15:56:40 +02:00
Sergei Golubchik
8cf589218f optimize performance of the build in a fresh clone
don't fetch the complete history of all submodules, it's rarely needed
2020-05-27 15:56:40 +02:00
Sergei Golubchik
cceb965a79 Revert "MDEV-12445 : Rocksdb does not shutdown worker threads and aborts in memleak check on server shutdown"
This reverts commit 6f1f911497.

because it doesn't do anything now (the server doesn't check
my_disable_leak_check) and it never did anything before
(because without `extern` it simply created a local instance of
my_disable_leak_check, did not affect server's my_disable_leak_check).
2020-05-27 15:56:40 +02:00
Sergei Golubchik
6af37ba881 fix rocksdb zstd detection
WITH_ROCKSDB_ZSTD must use the same capitalization as
in the foreach loop
2020-05-27 15:56:40 +02:00
Sergei Golubchik
ad77247866 MDEV-21958 Query having many NOT-IN clauses running forever and causing available free memory to use completely
let thd->killed to abort range optimizer
2020-05-27 15:56:40 +02:00
Sergei Golubchik
1e951155bd bugfix: use THD::main_mem_root for kill error message
cannot use the current THD::mem_root, because it can be temporarily
reassigned to something with a very different life time
(e.g. to TABLE::mem_root or range optimizer mem_root).
2020-05-27 15:56:40 +02:00
Sergei Golubchik
b01c8a6cc8 MDEV-22558 wrong error for invalid utf8 table comment 2020-05-27 15:56:40 +02:00
Sergei Golubchik
39c141b4ae don't include .git files in source packages
the existing ".git/" rule only filters out .git directories,
but as we have submodules now, we have to filter out .git files too.
2020-05-27 15:56:40 +02:00
Sergei Golubchik
a50e6c9eb1 MDEV-17153 server crash on repair table ... use_frm
data_file_length == 0 in mi_repair() is normal for REPAIR ... USE_FRM.
But in-file links (for blocks and deleted chain) must be compared with
the real file length to avoid spurious "link points outside datafile"
warnings and arbitrary block skipping.
2020-05-27 15:56:40 +02:00
Eugene Kosov
5139cfabb3 fix compilation 2020-05-27 16:37:34 +03:00
Monty
2347c18c79 Renamed maria_* source files for executables to aria_* 2020-05-27 15:47:10 +03:00
Monty
c52e62a76f Improve logging of Aria redo's and undo's
- Remove extra ',' and quotes
- Remove extra newline and remove double newlines
- Added options --lsn-redo-end and --lsn-undo-end to aria_read_log
- Allow one to give the aria_read_log lsn aruments as number,0xhexnumber,
  the same way as lsn's are written by aria_read_log
- Don't write full pages to redo log with EXTRA_DEBUG as this takes up
  a lot of disk and there has not been a need for this extra loggging for
  a long time. Instead one should use EXTRA_ARIA_DEBUG instead.
2020-05-27 15:47:10 +03:00
Monty
514533eb9a aria_chk sets --warning-if-wrong-transid=0 if --ignore-control-file is used 2020-05-27 15:47:10 +03:00
Marko Mäkelä
1b3adaab25 Fix the build with GCC 4.1.2
The build was broken on CentOS 5 and CentOS 6 as a result of
commit 18d8f06f31 introducing
some constructs that old GCC versions could not cope with.
2020-05-27 10:58:28 +03:00
Marko Mäkelä
bf1aa7569e Add an end-of-test marker to ease merges 2020-05-27 09:56:37 +03:00