Commit graph

70,273 commits

Author SHA1 Message Date
Sergei Golubchik
62d1e3bf67 MDEV-23569 temporary tables can overwrite existing files
for internal temporary tables: don't use realpath(),
and let them overwrite whatever orphan temp files might've
left in the tmpdir (see main.error_simulation test).

for user created temporary tables: we have to use realpath(),
(see 3a726ab6e2, remember DATA/INDEX DIRECTORY). don't allow
them to overwrite existing files.

This bug was reported by RACK911 LABS
2020-08-25 20:47:52 +02:00
Jan Lindström
0be70a1b77 MDEV-23483: Set Galera SST thd as system thread
Revert change to MDL and set SST donor thread as a system thread.
Joiner thread was already a system thread.
2020-08-25 12:12:44 +03:00
Aleksey Midenkov
6fa40b85be MDEV-23554 Wrong default value for foreign_key_checks variable
Sys_var_bit::session_save_default() ignored reverse_semantics property.
2020-08-25 10:15:04 +03:00
Andrei Elkin
a19cb3884f MDEV-23511 shutdown_server 10 times out, causing server kill at shutdown
Shutdown of mtr tests may be too impatient, esp on CI environment where
10 seconds of `arg` of `shutdown_server arg` may not be enough for the clean
shutdown to complete.

This is fixed to remove explicit non-zero timeout argument to
`shutdown_server` from all mtr tests. mysqltest computes 60 seconds default
value for the timeout for the argless `shutdown_server` command.
This policy is additionally ensured with a compile time assert.
2020-08-21 14:48:53 +03:00
Marko Mäkelä
bfba2bce6a Merge 10.1 into 10.2 2020-08-20 06:00:36 +03:00
Marko Mäkelä
4c50120d14 MDEV-23474 InnoDB fails to restart after SET GLOBAL innodb_log_checksums=OFF
Regretfully, the parameter innodb_log_checksums was introduced
in MySQL 5.7.9 (the first GA release of that series) by
mysql/mysql-server@af0acedd88
which partly replaced a parameter that had been introduced in 5.7.8
mysql/mysql-server@22ba38218e
as innodb_log_checksum_algorithm.

Given that the CRC-32C operations are accelerated on many processor
implementations (AMD64 with SSE4.2; since MDEV-22669 also on IA-32
with SSE4.2, POWER 8 and later, ARMv8 with some extensions)
and by lookup tables when only generic SISD instructions are available,
there should be no valid reason to disable checksums.

In MariaDB 10.5.2, as a preparation for MDEV-12353, MDEV-19543 deprecated
and ignored the parameter innodb_log_checksums altogether. This should
imply that after a clean shutdown with innodb_log_checksums=OFF one
cannot upgrade to MariaDB Server 10.5 at all.

Due to these problems, let us deprecate the parameter innodb_log_checksums
and honor it only during server startup.
The command SET GLOBAL innodb_log_checksums will always set the
parameter to ON.
2020-08-18 16:46:07 +03:00
Thirunarayanan Balathandayuthapani
8268f26605 MDEV-22934 Table disappear after two alter table command
Problem:
=======
InnoDB drops the column which has foreign key relations on it. So it
tries to load the foreign key during rename process of copy algorithm
even though the foreign_key_check is disabled.

Solution:
========
During alter copy algorithm, InnoDB ignores the error while loading
the foreign key constraint if foreign key check is disabled. It
should throw the warning about failure of the foreign key constraint
when foreign key check is disabled.
2020-08-18 15:05:23 +05:30
Daniel Black
b970363acf MDEV-23440: mysql_tzinfo_to_sql to use transactions
Since MDEV-18778, timezone tables get changed to innodb
to allow them to be replicated to other galera nodes.

Even without galera, timezone tables could be declared innodb.
With the standalone innodb tables, the mysql_tzinfo_to_sql takes
approximately 27 seconds.

With the transactions enabled in this patch, 1.2 seconds is
the approximate load time.

While explicit checks for the engine of the time zone tables could be
done, or checks against !opt_skip_write_binlog, non-transactional
storage engines will just ignore the transactional state without
even a warning so its safe to enact globally.

Leap seconds are pretty much ignored as they are a single insert
statement and have gone out of favour as they have caused MariaDB
stalls in the past.
2020-08-15 14:02:05 +10:00
Marko Mäkelä
182e2d4a6c Merge 10.1 into 10.2 2020-08-13 07:38:35 +03:00
Marko Mäkelä
101ce10d0d MDEV-20672 Inconsistent usage message for innodb_compression_algorithm
The usage message for the innodb_compression_algorithm system variable
did not list snappy, which was added as an optional compression algorithm
in MariaDB 10.1.3 and might actually work since
commit 90c52e5291 (MDEV-12615)
in MariaDB 10.1.24.

Unfortunately, we will include also unavailable compression algorithms
in the list, because ENUM parameters allow numeric values, and we do
not want innodb_compression_algorithm=3 to change meaning depending on
the way how the source code was compiled.
2020-08-12 18:35:21 +03:00
Marko Mäkelä
efd8af535a MDEV-19526 heap number overflow on innodb_page_size=64k
InnoDB only reserves 13 bits for the heap number in the record header,
limiting the heap number to be at most 8191. But, when using
innodb_page_size=64k and secondary index records of 7 bytes each,
it is possible to exceed the maximum heap number.

btr_cur_optimistic_insert(): Let the operation fail if the
maximum number of records would be exceeded.

page_mem_alloc_heap(): Move to the same compilation unit with the
only caller, and let the operation fail if the maximum heap number
has been allocated already.
2020-08-12 18:21:53 +03:00
Marko Mäkelä
c96be848d3 MDEV-14119 Assertion cmp_rec_rec() in ALTER TABLE
innobase_pk_order_preserved(): Treat an added AUTO_INCREMENT
column in the same way as an added existing column.
In either case, the column values are not guaranteed to
be constant, and thus the ordering may change if such a column
is added before any existing PRIMARY KEY columns.

prepare_inplace_alter_table_dict(): Initialize
dict_table_t::persistent_autoinc before invoking
innobase_pk_order_preserved().
2020-08-11 18:52:38 +03:00
Jan Lindström
57d1a5fa8e MDEV-22543 : Galera SST donation fails, FLUSH TABLES WITH READ LOCK times out
During SST we need to let FTWRL to use normal timeout method
even when client is disconnected.
2020-08-11 09:58:26 +03:00
Alexander Barkov
caf105905a Fixing sporading builtbot test failures happening at '00:00:00' sharp
Some tests relied on the fact that DATETIME->DATE conversion
always produce a truncation (with a warning). This is not the case
when the SQL statement is executed at current time '00:00:00' sharp.

Adding a new SET TIMESTAMP statements to make sure time is not '00:00:00'.
2020-08-11 10:33:10 +04:00
Marko Mäkelä
3b6dadb5eb Merge 10.1 into 10.2 2020-08-10 17:57:14 +03:00
Marko Mäkelä
0460d42b94 Merge mariadb-10.2.33 2020-08-10 17:31:49 +03:00
Daniel Black
deb365581b MDEV-23386: mtr: main.mysqld--help autosized table{-open,}-cach and max-connections
Example of the failure:
http://buildbot.askmonty.org/buildbot/builders/bld-p9-rhel7/builds/4417/steps/mtr/logs/stdio
```
main.mysqld--help 'unix'                 w17 [ fail ]
        Test ended at 2020-06-20 18:51:45

CURRENT_TEST: main.mysqld--help
--- /opt/buildbot-slave/bld-p9-rhel7/build/mysql-test/main/mysqld--help.result	2020-06-20 16:06:49.903604179 +0300
+++ /opt/buildbot-slave/bld-p9-rhel7/build/mysql-test/main/mysqld--help.reject	2020-06-20 18:51:44.886766820 +0300
@@ -1797,10 +1797,10 @@
 sync-relay-log-info 10000
 sysdate-is-now FALSE
 system-versioning-alter-history ERROR
-table-cache 421
+table-cache 2000
 table-definition-cache 400
-table-open-cache 421
-table-open-cache-instances 1
+table-open-cache 2000
+table-open-cache-instances 8
 tc-heuristic-recover OFF
 tcp-keepalive-interval 0
 tcp-keepalive-probes 0

mysqltest: Result length mismatch
```
mtr: table_open_cache_basic autosized:

Lets assume that >400 are available and that
we can set the result back to the start value.

All of these system variables are autosized and can
generate MTR output differences.

Closes #1527
2020-08-10 11:58:56 +02:00
Jan Lindström
845e3c9801 Replaced infinite loop in procedure with limited loop to avoid
hang.
2020-08-07 10:22:38 +03:00
Jan Lindström
1dec60c795 MDEV-22626: mysql_tzinfo_to_sql not replicates timezone to galeranodes if only 1 timezone will be loaded.
Move alter to InnoDB earlier to more correct place to handle
also if only a one timezone file is loaded.
2020-08-07 09:06:13 +03:00
Sergei Golubchik
fbcae42c2a Merge branch '10.1' into 10.2 2020-08-06 16:47:39 +02:00
Varun Gupta
caa474f8e3 MDEV-15180: server crashed with NTH_VALUE()
fix_fields for the arguments of the NTH_VALUE function was updating the same reference,
so for the second argument (or after the first argument) the items were not resolved
to their corresponding field from the view as they were updating the reference to the
first argument.
2020-08-06 18:01:00 +05:30
Varun Gupta
ab578bdf45 MDEV-9513: Assertion `join->group_list || !join->is_in_subquery()' failed in create_sort_index
Removing the ORDER BY clause from the UNION when UNION is inside an IN/ALL/ANY/EXISTS subquery.
The rewrites are done for subqueries but this rewrite is not done for the fake_select of
the UNION.
2020-08-06 10:55:03 +05:30
Varun Gupta
1e31d74833 MDEV-17066: Bytes lost or Assertion `status_var.local_memory_used == 0 after DELETE with subquery with ROLLUP
The issue here is when records are read from the temporary file
(filesort result in this case) via a cache(rr_from_cache).
The cache is initialized with init_rr_cache.
For correlated subquery the cache allocation is happening at each execution
of the subquery but the deallocation happens only once and that was
when the query execution was done.

So generally for subqueries we do two types of cleanup

1) Full cleanup: we should free all resources of the query(like temp tables).
   This is done generally when the query execution is complete or the subquery
   re-execution is not needed (case with uncorrelated subquery)

2) Partial cleanup: Minor cleanup that is required if
   the subquery needs recalculation. This is done for all the structures that
   need to be allocated for each execution (example SORT_INFO for filesort
   is allocated for each execution of the correlated subquery).

The fix here would be free the cache used by rr_from_cache in the partial
cleanup phase.
2020-08-05 11:28:32 +05:30
Alexander Barkov
0e80f5a693 MDEV-23105 Cast number string with many leading zeros to decimal gives unexpected result
Skip leading zeros when converting a string to decimal_t.
2020-08-05 08:14:49 +04:00
Sachin
e3c18b8e84 MDEV-23089 rpl_parallel2 fails in 10.5
Problem:- rpl_parallel2 was failing non-deterministically
Analysis:-
When FLUSH TABLES WITH READ LOCK is executed, it will allow all worker
threads to complete their ongoing transactions and then it will pause them.
At this state FTWRL will proceed to acquire global read lock. FTWRL first
blocks threads from starting new commits, then upgrades the lock to block
commit of existing transactions.
  Step1:
    FLUSH TABLES WITH READ LOCK - Blocks new commits
  Step2:
    * STOP SLAVE command enables 'force_abort=1' which unblocks workers,
      they continue to execute events.
    * T1: Waits in 'record_gtid' call to update 'gtid_slave_pos' table with
      its current GTID, but it is blocked becuase of Step1.
    * T2: Holds COMMIT lock and waits for T1 to commit.
  Step3:
    FLUSH TABLES WITH READ LOCK - Waiting to get BLOCK_COMMIT.
This results in deadlock. When STOP SLAVE command allows paused workers to
proceed, workers should skip the execution of all further events, similar
to 'conservative' parallel mode.
Solution:-
We will assign 1 to skip_event_group when we are aborted in do_ftwrl_wait.
rpl_parallel_entry->pause_sub_id is only reset when force_abort is off in
rpl_pause_after_ftwrl.
2020-08-04 11:28:26 +05:30
Rucha Deodhar
5fb07d22f1 MDEV-23082: ER_TABLEACCESS_DENIED_ERROR error message is truncated,
and inaccurately

Analysis: The list of all privileges is 118 characters wide. However, the
format of error message was: "%-.32s command denied to user...". get_length()
sets the maximum width to 32 characters. As a result, only first 32
characters of list of privilege are stored.
Fix: Changing the format to "%-.100T..." so that get_length() sets width to
100. Hence, first 100 characters of the list of privilege are stored and the
type specifier 'T' appends '...' so that truncation can be seen.
2020-08-04 10:55:21 +05:30
Rucha Deodhar
745fa255ba MDEV-14836: Assertion `m_status == DA_ERROR' failed in
Diagnostics_area::sql_errno upon query from I_S with LIMIT ROWS EXAMINED

open_normal_and_derived_table() fails because the query was already killed
as rows examined by the query are more than the limit. However, this isn't a
real error.
Fix: Check if there is actually an error before calling thd->sql_errno()
and later send a warning in handle_select() if no real error.
2020-08-04 10:55:21 +05:30
Jan Lindström
87b1625b5c Test case MW-328A still fails, thus disable it until it is really
fixed.
2020-08-04 07:53:13 +03:00
Elena Stepanova
dc716da457 List of unstable tests for 10.2.33 release 2020-08-03 03:34:57 +03:00
Elena Stepanova
9c84b80f84 Merge branch '10.1' into 10.2 2020-08-03 02:57:50 +03:00
Elena Stepanova
dbb4572fe9 MDEV-23375 parts.partition_debug fails when it's run after another test
Make sure system tables aren't open, as the test kills the server
2020-08-03 02:42:07 +03:00
Elena Stepanova
7a4287d421 List of unstable tests for 10.1.46 release 2020-08-02 20:19:53 +03:00
Oleksandr Byelkin
ef7cb0a0b5 Merge branch '10.1' into 10.2 2020-08-02 11:05:29 +02:00
Thirunarayanan Balathandayuthapani
a6066e230e MDEV-22511 innodb.truncate_foreign failed in buildbot with wrong error code
- Adding lock_wait_timeout value as 1 make sure that truncate table
fails instead of making MDL timeout.
2020-07-31 15:07:43 +05:30
Marko Mäkelä
f35d172103 MDEV-23198 Crash in REPLACE
row_vers_impl_x_locked_low(): clust_offsets may point to memory
that is allocated by mem_heap_alloc() and may have been freed.
For initializing clust_offsets, try to use the stack-allocated
buffer instead of a pointer that may point to freed memory.

This fixes a regression that was introduced in
commit f0aa073f2b (MDEV-20950).
2020-07-31 11:54:35 +03:00
Nikita Malyavin
fd0abc890f MDEV-18042 Server crashes upon adding a non-null date column under NO_ZERO_DATE with ALGORITHM=INPLACE
accept table_name and db_name instead of table_share in make_truncated_value_warning
2020-07-31 17:38:41 +10:00
Nikita Malyavin
91ebf1844f MDEV-19338 InnoDB: Failing assertion: !cursor->index->is_committed()
Call mark_columns_per_binlog_row_image before find_row() to set up table->vcol_set early,
so the virtual column value will be updated after record read (ha_rnd_pos/ha_index_next/etc)
by table->update_virtual_fields() call
2020-07-31 17:32:29 +10:00
Sergei Golubchik
09ec8e2e22 improve the error message for a dropped current role 2020-07-30 23:50:56 +02:00
Sergei Golubchik
4635218cb0 MDEV-22521 Server crashes in traverse_role_graph_up or Assertion `user' fails in traverse_role_graph_impl 2020-07-30 23:50:56 +02:00
Marko Mäkelä
4860fe244b XtraDB 5.6.49-89.0
The only change between Percona XtraDB Server 5.6.48-88.0
and 5.6.49-89.0 (apart from the version number change) was
percona/percona-server@25ec240920
which we had already addressed in
commit 7c03edf2fe and
commit c0fca2863b.
2020-07-30 11:06:46 +03:00
Marko Mäkelä
c5d4dd2533 MDEV-23339 innodb_force_recovery=2 may still abort the rollback of recovered transactions
trx_rollback_active(), trx_rollback_resurrected(): Replace
an incorrect condition that we failed to replace in
commit b68f1d847f (MDEV-21217).
2020-07-30 09:24:36 +03:00
Sergei Petrunia
7e9ffc69ec MDEV-21472: ALTER TABLE ... ANALYZE PARTITION ... with EITS reads and locks all rows
Do not collect EITS statistics for this statement:

  ALTER TABLE t ANALYZE PARTITION p

EITS stats are currently global, not per-partition.
Collecting global stats when we are asked to process just one partition
causes issues for DBAs.
2020-07-29 23:27:40 +03:00
Sergei Golubchik
e54a7ac1b3 MDEV-23010 UPDATE privilege at Database and Table level fail to update with SELECT command denied to user
check both column- and table-level grants when looking for SELECT
privilege on UPDATE statement.
2020-07-29 14:56:24 +02:00
Sergei Golubchik
0b5b2f8641 Bug #25207522: INCORRECT ORDER-BY BEHAVIOR ON A PARTITIONED TABLE WITH A COMPOSITE PREFIX INDEX
Fix prefix key comparison in partitioning. Comparions must
take into account no more than prefix_len characters.

It used to compare prefix_len*mbmaxlen bytes.
2020-07-29 14:56:24 +02:00
Sergei Golubchik
77b7f793f9 MDEV-18496 Crash when Aria encryption is enabled but plugin not available
fix uninitialized struct member
2020-07-29 14:56:24 +02:00
Sergei Golubchik
88cbe2f081 MDEV-18496 Crash when Aria encryption is enabled but plugin not available
wait_while_table_is_used() should return an error if handler::extra() fails
2020-07-29 14:56:24 +02:00
Oleksandr Byelkin
2107e3bb9c MDEV-21258: Can't uninstall plugin if the library file doesn't exist
Removing plugin from the mysql.plugin even if the plugin is not loaded
2020-07-29 12:34:53 +02:00
Marko Mäkelä
6307b17aa1 MDEV-20142 encryption.innodb_encrypt_temporary_tables failed in buildbot with wrong result
Let us read both encrypted temporary tables to increase the changes of
page flushing and eviction.
2020-07-28 13:00:59 +03:00
Dan Solodko
459b87f6b4 MDEV-9911: NTILE must return an error when parameter is not stable 2020-07-28 06:35:32 +02:00
Daniel Black
beec8404fa MDEV-17076: mtr int options aren't negative 2020-07-28 09:08:36 +10:00