Commit graph

193035 commits

Author SHA1 Message Date
Vladislav Vaintroub
3d6eb7afcf MDEV-25602 get rid of __WIN__ in favor of standard _WIN32
This fixed the MySQL bug# 20338 about misuse of double underscore
prefix __WIN__, which was old MySQL's idea of identifying Windows
Replace it by _WIN32 standard symbol for targeting Windows OS
(both 32 and 64 bit)

Not that connect storage engine is not fixed in this patch (must be
fixed in "upstream" branch)
2021-06-06 13:21:03 +02:00
Marko Mäkelä
06dd151bb8 Merge 10.5 into 10.6 2021-06-02 11:31:01 +03:00
Vladislav Vaintroub
3dd91241cb MDEV-24985 Shutdown fails to abort current InnoDB lock waits
On shutdown, to kill remaining connections, do the same thing that
server does during KILL CONNECTION, i.e thd->awake().

The stripped-down KILL version, that was used prior to this patch
for shutdown, missed the engine specific part (ha_kill_query)
2021-06-02 09:04:00 +02:00
Marko Mäkelä
8426c7411b MDEV-23399 fixup for MDEV-10814
buf_madvise_do_dump(): Fix a mutex release that was broken in
commit 7cffb5f6e8.
This function is not covered by any tests. Its only purpose is
to be called from a debugger so that buffers that would normally
be excluded from core dumps ever since
commit b600f30786
can be included.
2021-06-02 08:48:09 +03:00
Marko Mäkelä
49ab50f882 Merge 10.4 into 10.5 2021-06-02 08:30:33 +03:00
Marko Mäkelä
d3d2c96567 Merge 10.3 into 10.4 2021-06-02 08:29:47 +03:00
Marko Mäkelä
aa70690e9a Merge 10.2 into 10.3 2021-06-02 08:29:01 +03:00
Marko Mäkelä
4847a2e730 MDEV-17749 fixup: ./mtr --embedded main.lock_kill 2021-06-02 08:28:48 +03:00
Marko Mäkelä
a8434c6c59 MDEV-25730 fixup: GCC -Og -Wmaybe-uninitialized
Silence a warning about an uninitialized variable that was
introduced by commit d8fa71a089.
2021-06-02 08:25:44 +03:00
Monty
6d21b6acb8 Updated main.trigger-trans.test to make it more resiliant 2021-06-02 01:03:38 +03:00
Marko Mäkelä
4555fb1722 After-merge fix: ./mtr --embedded main.lock_kill 2021-06-01 15:46:52 +03:00
Marko Mäkelä
a722ee88f3 Merge 10.5 into 10.6 2021-06-01 11:39:38 +03:00
Marko Mäkelä
9c7a456a92 Merge 10.4 into 10.5 2021-06-01 10:38:09 +03:00
Marko Mäkelä
77d8da57d7 Merge 10.3 into 10.4 2021-06-01 09:14:59 +03:00
Marko Mäkelä
950a220060 Merge 10.2 into 10.3 2021-06-01 08:40:59 +03:00
Daniel Black
5727d56260 Change connection_count back to static
Code structual change only, just limiting
super global variables especially when the
counter under the THD structure should be used
in general.
2021-06-01 13:53:16 +10:00
Krunal Bauskar
2a4f72b7d2 MDEV-25807: ARM build failure due to missing ISB instruction on ARMv6
Debian has support for 3 different arm machine and probably one which is
failing is a pretty old version which doesn't support the said instruction.

So we can make it specific to _aarch64_ (as per the arm official reference
manual ISB should be defined by all ARMv8 processors). [as per the ARMv7 specs
even 32 bits processor should support it but not sure which exact version
Debian has under armel].

In either case, the said performance issue will have an impact mainly with a
high-end processors with extreme parallelism so it safe to limit it to _aarch64_.

Corrects: MDEV-24630
2021-06-01 13:51:39 +10:00
Daniel Black
0d44792a83 perfschema: native type for my_thread_os_id_t
Though these will all get case to unsigned long
long where it is populated into the perfschema's BIGINT
type.

Use uintptr_t for NetBSD per Nia Alarie's original #1836.
2021-06-01 13:51:39 +10:00
Daniel Black
90adf2aa59 perfschema: use glibc gettid if available 2021-06-01 13:51:39 +10:00
nia
68eac8a3ad my_thread: Use unsigned long long for storing pthread IDs
This is a fix for operating systems that have pthread_t defined
as a pointer and use the default pthread_self() mechanism for
identifying threads. More specifically, this is a build fix
for NetBSD.

Any changes I submit are freely available under the new BSD
license.

Signed-off-by: Nia Alarie <nia@NetBSD.org>
2021-06-01 13:51:39 +10:00
Julius Goryavsky
2fb4407827 MDEV-25818: RSYNC SST failed due to busy port
This commit reduces the likelihood of getting a busy port on
quick restarts with rsync SST (problem MDEV-25818) and fixes
a number of other flaws in SST scripts, adds new functionality,
and also synchronizes the xtrabackup-v2 script with the
mariabackup script (the latter applies only to the 10.2 branch):

 1) SST via rsync: rsync and stunnel does not always get the right
    time to complete by correctly handling SIGTERM. These utilities
    are now given more time to complete normally (via normal SIGTERM
    processing) before we move on to using "kill -9";
 2) SST via rsync: attempts to terminate an rsync or stunnel process
    (via "kill" utility) are only made if it did not terminated on
    its own;
 3) SST via rsync: if a combination of stunnel and rsync is used,
    then we need to wait for both utilities to finish or stop, not
    just one of them;
 4) The config file and pid file for stunnel are now deleted after
    successful completion of SST on the donor node;
 5) The configs and pid files from rsync and stunnel should not be
    deleted unless these utilities succeed (or are sucessfully
    terminated) on the joiner node;
 6) The configs and pid files now excluded from transfer via rsync;
 7) Spaces in paths are now valid for config files as well (when
    used with SST via rsync or mariabackup / xtrabackup[-v2]);
 8) SST via mariabackup: added preliminary verification of keys and
    certificates that are used when establishing a connection using
    SSL (to avoid long timeouts and improve diagnostics) - by analogy
    with how it is done for the xtrabackup-v2 (plus check for CA file),
    while that check is skipped if the user does not have openssl
    installed (or does not have diff utility);
 9) Added backup-threads=<n> configuration option which adds
    "--parallel=<n>" for mariabackup / xtrabackup at backup and
    move-back stages;
10) Added encrypt-threads and encrypt-chunk-size configuration
    options for xbcrypt management (when xbcrypt is used);
11) Small optimization: checking the socat version and adding
    a file with parameters for 2048-bit Diffie-Hellman (if necessary)
    is done only if the user has not specified "dhparam=" in the
    "sockopt" option value;
12) SST via rsync now supports "backup-threads" configuration option
    (in server-related sections or in the "[sst]");
13) Determining the number of available processors is now supported
    for FreeBSD + mariabackup/xtrabackup: before that we might have
    problems with "--compact" (rebuild indexes) or qpress on FreeBSD;
14) The check_pid() function should not raise an error state in
    the rare cases when the pid file was created, but it is empty,
    or if it is deleted right during the check, or when zero is read
    from the pid file;
15) Iproved templates that are used to check if a requested socket
    is "listening" when using the ss utility;
16) Shortened some other templates for socket state utilities;
17) Temporary files created by mariabackup / xtrabackup are moved
    to a separate subdirectory inside tmpdir (so they don't get
    mixed with other temporary files, which can make debugging
    more difficult);
18) 10.2 only: the script for SST via xtrabackup-v2 has been brought
    in full compliance with all the bugfixes made for mariabackup (as
    it previously contained many flaws compared to the updated script
    for mariabackup).
2021-05-31 14:56:35 +02:00
Marko Mäkelä
139333a6cc MDEV-25745: Not applying INSERT_REUSE_REDUNDANT
page_apply_insert_redundant(): Correct a condition that would
occasionally fail when recovering changes for the change buffer tree
(where extra_size and data_size can vary wildly).

This was broken in commit 138cbec5f2
(MDEV-21724).
2021-05-31 15:44:11 +03:00
Marko Mäkelä
6ca065468f MDEV-19514 fixup: Optimize ibuf_merge_or_delete_for_page() calls 2021-05-31 15:44:04 +03:00
Marko Mäkelä
601eb41183 Cleanup: deduplicate code 2021-05-31 15:44:04 +03:00
Alexey Botchkov
b118f92be6 MDEV-15888 Implement FLUSH TABLES tbl_name [, tbl_name] ... WITH READ LOCK for views.
Enable the FLUSH TABLES for views. It works now.
2021-05-31 16:18:45 +04:00
Vladislav Vaintroub
d3c77e08ae MDEV-20556 Remove references to "xtrabackup" and "innobackupex" in mariabackup --help 2021-05-31 12:54:21 +02:00
Georg Richter
be8e51c459 MDEV-25511: Command line tools don't support CRL parameters
Enable CRL support for GnuTLS (which was implemented by CONC-433).
2021-05-31 08:29:37 +02:00
Dmitry Shulga
91bde0fb67 MDEV-25576: The statement EXPLAIN running as regular statement and as prepared statement produces different results for UPDATE with subquery
Both EXPLAIN and EXPLAIN EXTENDED statements produce different results set
in case it is run in normal way and in PS mode for the statements
UPDATE/DELETE with subquery.

The use case below reproduces the issue:
MariaDB [test]> CREATE TABLE t1 (c1 INT KEY) ENGINE=MyISAM;
Query OK, 0 rows affected (0,128 sec)

MariaDB [test]> CREATE TABLE t2 (c2 INT) ENGINE=MyISAM;
Query OK, 0 rows affected (0,023 sec)

MariaDB [test]> CREATE TABLE t3 (c3 INT) ENGINE=MyISAM;
Query OK, 0 rows affected (0,021 sec)

MariaDB [test]> EXPLAIN EXTENDED UPDATE t3 SET c3 =
    -> ( SELECT COUNT(d1.c1) FROM ( SELECT a11.c1 FROM t1 AS a11
    -> STRAIGHT_JOIN t2 AS a21 ON a21.c2 = a11.c1 JOIN t1 AS a12
    -> ON a12.c1 = a11.c1 ) d1 );
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                          |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
|    1 | PRIMARY     | t3    | ALL  | NULL          | NULL | NULL    | NULL |    0 |   100.00 |                                |
|    2 | SUBQUERY    | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | Impossible WHERE noticed after reading const tables
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
2 rows in set (0,002 sec)

MariaDB [test]> PREPARE stmt FROM
    -> EXPLAIN EXTENDED UPDATE t3 SET c3 =
    -> ( SELECT COUNT(d1.c1) FROM ( SELECT a11.c1 FROM t1 AS a11
    -> STRAIGHT_JOIN t2 AS a21 ON a21.c2 = a11.c1 JOIN t1 AS a12
    -> ON a12.c1 = a11.c1 ) d1 );
Query OK, 0 rows affected (0,000 sec)
Statement prepared

MariaDB [test]>  EXECUTE stmt;
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                          |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
|    1 | PRIMARY     | t3    | ALL  | NULL          | NULL | NULL    | NULL |    0 |   100.00 |                                |
|    2 | SUBQUERY    | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL |     NULL | no matching row in const table |
+------+-------------+-------+------+---------------+------+---------+------+------+----------+--------------------------------+
2 rows in set (0,000 sec)

The reason by that different result sets are produced is that on execution
of the statement 'EXECUTE stmt' the flag SELECT_DESCRIBE not set
in the data member SELECT_LEX::options for instances of SELECT_LEX that
correspond to subqueries used in the UPDTAE/DELETE statements.

Initially, these flags were set on parsing the statement
  PREPARE stmt FROM "EXPLAIN EXTENDED UPDATE t3 SET ..."
but latter they were reset before starting real execution of
the parsed query during handling the statement 'EXECUTE stmt';

So, to fix the issue the functions mysql_update()/mysql_delete()
have been modified to set the flag SELECT_DESCRIBE forcibly
in the data member SELECT_LEX::options for the primary SELECT_LEX
of the UPDATE/DELETE statement.
2021-05-30 17:31:55 +07:00
Vladislav Vaintroub
5bd517259f MDEV-25815 mariabackup crash or debug assert with --backup --databases-exclude
Fix regression (debug assertion or division by 0)
caused by cfd3d70ccb
2021-05-29 06:32:40 +02:00
Anel Husakovic
904edfd24b MDEV-25777: JAVA_INCLUDE_PATH and JAVA_INCLUDE_PATH2 not found with out of source configuration and Ninja generator
- As solution `PLUGIN_CONNECT=NO` use early check to disable plugin:
  Solution suggested by wlad@mariadb.com
- `JNI_FOUND` is a internal result variable and should be set with
cached library and header variables (like `JAVA_INCLUDE_PATH`) defined.
  * Note: wrapper cmake/FindJNI.cmake runs first time and cmake native Find<module> returns only cached variable, like `JAVA_INCLUDE_PATH`, results variable are not cached).

Reviewed by: serg@mariadb.com
2021-05-28 21:03:48 +02:00
Otto Kekäläinen
a70a5537e7 Deb: Innotop: Add support for MariaDB 10.5+
Synced from downstream Debian:
7015e8e4b5
2021-05-28 14:04:10 +10:00
Sergei Golubchik
d06205ba37 CONNECT: use my_snprintf 2021-05-27 17:36:32 +02:00
Sergei Golubchik
1638241e31 mtr: fix the debug printout
to print all arguments of _verbose(), not just the number of them
2021-05-27 17:36:32 +02:00
Sergei Golubchik
08bc7ee068 MDEV-25792 server hangs on early shutdown if InnoDB needs to purge indexed virtual columns
if the server isn't started and innodb initiated a shutdown process -
don't wait for the server to start, it won't
2021-05-27 16:46:21 +02:00
Robert Bindar
2274c4f5fc Add Pull Request template file to the MariaDB/server repository 2021-05-27 16:28:28 +03:00
Robert Bindar
ef0d883903 Revert "Add Pull Request template file to the MariaDB/server repository"
This reverts commit 17106c984b.
2021-05-27 15:43:31 +03:00
Marko Mäkelä
f078788e29 MDEV-25312 fixup: Fix .isl file parsing
read_link_file(): Avoid GCC -Wtype-limits when char is unsigned,
by always using unsigned comparison. Also, avoid buffer overflow
when the .isl file consists entirely of spaces or control codes.
2021-05-27 13:28:08 +03:00
Marko Mäkelä
a7d68e7a0f MDEV-25791: Remove UNIV_INTERN
Back in 2006 or 2007, when MySQL AB and Innobase Oy existed as
separately controlled entities (Innobase had been acquired by
Oracle Corporation), MySQL 5.1 introduced a storage engine plugin
interface and Oracle made use of it by distributing a separate
InnoDB Plugin, which would contain some more bug fixes and
improvements, compared to the version of InnoDB that was statically
linked with the mysqld server that was distributed by MySQL AB.
The built-in InnoDB would export global symbols, which would clash
with the symbols of the dynamic InnoDB Plugin (which was supposed
to override the built-in one when present).

The solution to this problem was to declare all global symbols with
UNIV_INTERN, so that they would get the GCC function attribute that
specifies hidden visibility.

Later, in MariaDB Server, something based on Percona XtraDB (a fork of
MySQL InnoDB) became the statically linked implementation, and something
closer to MySQL InnoDB was available as a dynamic plugin. Starting with
version 10.2, MariaDB Server includes only one InnoDB implementation,
and hence any reason to have the UNIV_INTERN definition was lost.

btr_get_size_and_reserved(): Move to the same compilation unit with
the only caller.

innodb_set_buf_pool_size(): Remove. Modify innobase_buffer_pool_size
directly.

fil_crypt_calculate_checksum(): Merge to the only caller.

ha_innobase::innobase_reset_autoinc(): Merge to the only caller.

thd_query_start_micro(): Remove. Call thd_start_utime() directly.
2021-05-27 13:28:08 +03:00
Thirunarayanan Balathandayuthapani
c11c5f36d8 MDEV-25758 InnoDB spatial indexes miss large geometry fields after MDEV-25459
InnoDB should calculate the MBR for the first field of
spatial index and do the comparison with the clustered
index field MBR. Due to MDEV-25459 refactoring, InnoDB
calculate the length of the first field and fails with
too long column error.
2021-05-27 15:01:19 +05:30
Marko Mäkelä
ab87fc6c7a Cleanup: Remove handler::update_table_comment()
The only call of the virtual member function
handler::update_table_comment() was removed in
commit 82d28fada7 (MySQL 5.5.53)
but the implementation was not removed.

The only non-trivial implementation was for InnoDB. The information
is now returned via handler::get_foreign_key_create_info() and
ha_statistics::delete_length.
2021-05-27 09:31:19 +03:00
Sergei Golubchik
4777097fee followup: rename generated files to have distinct names 2021-05-27 00:40:23 +02:00
Sergei Golubchik
dfbeddaa11 MDEV-25726 get rid of cmake comment hack in sql_yacc.yy
and replace it with equally unsightly %ifdef/%endif hack
also, support %else, it's nice
2021-05-27 00:40:23 +02:00
Monty
9ec2129f71 Fixed bug in mtr that caused restart to fail if mysqld died to fast 2021-05-26 22:17:51 +03:00
Monty
552bb1af7b Removed test for MDEV-23842 as the results are not stable 2021-05-26 22:12:49 +03:00
Monty
e8a54a376a Replace item->marker= # with Item->marker= DEFINE
- Better, easier to read code (no used of 'random' constants).
- All defines are now unique, so it is easier to find bugs if
  somethings goes wrong.

Other things:
- Created sub function of common code in Aggregator_distinct::setup() and
  Item_func_group_concat::setup() that set item->marker
- More documentation
- Folded a few long lines.
- Allmost all changes in item.cc, sql_lex.cc and sql_window.cc are done
  with 'replace'.
2021-05-26 22:12:49 +03:00
Monty
1e5ebf3762 Fixed results for main.delete_use_source to make it repeatable 2021-05-26 14:53:26 +03:00
Monty
aa284e0237 MDEV-17749 Kill during LOCK TABLE ; ALTER TABLE causes assert
The problem was that when LOCK TABLES where unwinded as part of
a killed connection, unlink_all_closed_tables() did not like that
there was uncommited transactions.
Fixed by doing a rollback of any open transaction in this particular case.
2021-05-26 14:35:23 +03:00
Robert Bindar
17106c984b Add Pull Request template file to the MariaDB/server repository 2021-05-26 12:00:30 +03:00
Jan Lindström
288b801696 Fix MDEV-25562 test case. 2021-05-26 11:54:29 +03:00
Marko Mäkelä
860e754349 Merge 10.5 into 10.6 2021-05-26 11:22:40 +03:00