Commit graph

191580 commits

Author SHA1 Message Date
Monty
fc5e03f093 Ignore reporting in thd_progress_report() if we cannot lock LOCK_thd_data
The reason for this is that Galera can lock LOCK_thd_data for a long time.

Instead of stalling any long running process, like alter or repair table,
because of progress reporting, ignore the progress reporting for this
call. Progress reporting will continue on the next call after the lock has
been released.
2021-02-15 20:21:29 +02:00
Aleksey Midenkov
8eaf4bc5ec Comment on assertion in row_rename_table_for_mysql()
Related to commit f0baa864 and MDEV-23632.
2021-02-15 13:57:37 +03:00
Monty
34c654024c MDEV-24855 ER_CRASHED_ON_USAGE or Assertion `length <= column->length'
When creating a summary temporary table with bit fields used in the sum
expression with several parameters, like GROUP_CONCAT(), the counting of
bits needed in the record was wrong.

The reason we got an assert in Aria was because the bug caused a memory
overwrite in the record and Aria noticed that the data was 'impossible.
2021-02-15 01:33:06 +02:00
Jan Lindström
b3df194e31 MDEV-24833 : Signal 11 on wsrep_can_run_in_toi at wsrep_mysqld.cc:1994
Problem was that when engine substitution is allowd (e.g. sql_mode='')
we must also check db_type. Additionally, we did not resolve
default storage engine on that case and used that to check is
TOI possible or not.
2021-02-12 20:43:03 +02:00
Otto Kekäläinen
7fb528d722 Deb: Remove build depencency libreadline-gplv2-dev no longer available
The Readline library is no longer available in Debian Sid.
See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=980504

Add the dependency on-the-fly in autobake-deb.sh for older distro versions
and keep the native build in a state that works on Debian Sid as-is.
2021-02-11 17:59:09 +02:00
Otto Kekäläinen
2405752855 Salsa-CI: Install readline from Buster as it was removed from Sid
The older MariaDB.org packages were built against libreadline5 and depend
on it to install. As it was removed from Sid (and Bullseye), fetch and
install it from Buster.

This can eventually be removed once we stop backwards compat testing for
older MariaDB versions (new ones don't depend on it).

Upstreamed from Debian packaging repository commit
4b729db7f8
2021-02-11 17:59:09 +02:00
Daniel Black
c7edbe5bb1 MDEV-24366: s3 test postfix - use default for S3_BUCKET
and S3_HOST_NAME.

Required environment variables are now S3_ACCESS_KEY and S3_SECRET_KEY.

Or a running minio instance on localhost:9000.
2021-02-11 12:51:29 +11:00
Daniel Black
5e3d3220bb MDEV-24344: BINLOG REPLAY privilege is missing from SHOW PRIVILEGES
Was added in 10.5.2 (MDEV-21975)
2021-02-09 19:41:00 +11:00
Monty
ffc5d06489 MDEV-24087 s3.replication_partition fails in buildbot wiht replication failure
A few of the failures was because of missing sync_slave_to_master in
the test suite.

However, the biggest reason for most faulures was that in case of
ALTER PARTITION the master writes the query to the binary log before
it has updated the .frm and .par files. This causes a problem for an
S3 slave as it will start execute the ALTER PARTITION but get old .frm and
.par files from S3 which causes "open table" to fail, either with an error
or in some case with a crash.
Fixed
2021-02-08 21:03:04 +02:00
Monty
bd5ac03896 Make maria_data_root const char*
This allow one to remove some casts like:
maria_data_root= (char *)".";

It also removes warnings from icc.
2021-02-08 12:16:29 +02:00
Monty
5d6ad2ad66 Added 'const' to arguments in get_one_option and find_typeset()
One should not change the program arguments!
This change also reduces warnings from the icc compiler.

Almost all changes are just syntax changes (adding const to
'get_one_option function' declarations).

Other changes:
- Added a few cast of 'argument' from 'const char*' to 'char *'. This
  was mainly in calls to 'external' functions we don't have control of.
- Ensure that all reset of 'password command line argument' are similar.
  (In almost all cases it was just adding a comment and a cast)
- In mysqlbinlog.cc and mysqld.cc there was a few cases that changed
  the command line argument. These places where changed to instead allocate
  the option in a MEM_ROOT to avoid changing the argument. Some of this
  code was changed to ensure that different programs did parsing the
  same way. Added a test case for the changes in mysqlbinlog.cc
- Changed a few variables that took their value from command line options
  from 'char *' to 'const char *'.
2021-02-08 12:16:29 +02:00
Monty
e30a3048da Ensure that mysqlbinlog frees all memory at exit 2021-02-08 12:12:22 +02:00
Marko Mäkelä
4f4a4cf9eb MDEV-23399 fixup: Use plain pthread_cond
The condition variables that were introduced in
commit 7cffb5f6e8 (MDEV-23399)
are never instrumented with PERFORMANCE_SCHEMA.
Let us avoid the storage overhead and dead code.
2021-02-07 12:19:24 +02:00
Marko Mäkelä
7ce643782b MDEV-23379 fixup: Remove dead PERFORMANCE_SCHEMA code 2021-02-07 12:11:16 +02:00
Monty
eacefbca35 MDEV-24750 Various corruptions caused by Aria subsystem...
The test case was setting aria_sort_buffer_size to MAX_ULONGLONG-1
which was not handled gracefully by my_malloc() or safemalloc().
Fixed by ensuring that the malloc functions returns 0 if the size
is too big.
I also added some protection to Aria repair:
- Limit sort_buffer_size to 16G (after that a bigger sort buffer will
  not help that much anyway)
- Limit sort_buffer_size also according to sort file size. This will
  help by not allocating less memory if someone sets the buffer size too
  high.
2021-02-03 02:22:47 +02:00
Thirunarayanan Balathandayuthapani
b76e5c6610 MDEV-24765 fseg_free_extent fails to call buf_page_free() for the whole segment
This is caused by commit c92f7e287fc0e21dc1b181284b1f8e2139d1c331(MDEV-8139).
InnoDB fails to set the page status as FREED in buffer pool while freeing
the extent.
2021-02-02 18:17:44 +05:30
Vladislav Vaintroub
b5dab19efa MDEV-24757 : fix potential null pointer dereference in I_S.thread_pool_queues
With the fix, new connection without THDs will now show NULL in
thread_pool_queues.CONNECTION_ID.
2021-02-02 00:19:52 +01:00
Sergei Golubchik
e2c571ad84 MDEV-24498 mysql_config reports wrong libs for libmysqld
filter out tpool
2021-02-01 20:05:54 +01:00
Marko Mäkelä
324e5f02a9 MDEV-24754 Crash in ha_partition_inplace_ctx::~ha_partition_inplace_ctx()
ha_innobase::commit_inplace_alter_table(): Fix a regression that was
introduced in 6d1f1b61b5 (MDEV-24564).
2021-02-01 18:45:35 +02:00
David CARLIER
b1241585b2
Mac M1 build support proposal (minus rocksdb option) () 2021-01-30 17:04:27 +02:00
FX Coudert
52795c2f78 Apple Silicon is a 64-bit platform 2021-01-30 16:35:46 +02:00
Sergei Petrunia
73c43ee9ed MDEV-24739: Assertion `root->weight >= ...' failed in SEL_ARG::tree_delete
Also update the SEL_ARG graph weight in:
- sel_add()
- SEL_ARG::clone()

Make key_{and,or}_with_limit() to also verify weight for the arguments
(There is no single point to verify SEL_ARG graphs constructed from
conditions that are not AND-OR formulas, so we hope that those are
connected with AND/OR and do it here).
2021-01-30 00:54:21 +03:00
Marko Mäkelä
a70a47f2f3 MDEV-24661: Remove the test innodb.innodb_wl6326_big
The purpose of the test was to ensure that the SX (update) mode of
index tree and buffer page latches are being used.

The test has become unstable, possibly due to changes related to
buf_pool.mutex and buf_pool.page_hash, or to the use of MDL in the
purge of transaction history.

In 10.6, the test depends on instrumentation that was refactored
or removed in MDEV-24142.

The use of different latching modes can better be indirectly observed
through high-concurrency benchmarks. For MDEV-14637, a performance test
was conducted where the finer-grained latching and
BTR_CUR_FINE_HISTORY_LENGTH were removed. It caused a 20% performance
regression for UPDATE and somewhat smaller for INSERT.

Any new problem with latching granularity should be easily caught by
performance testing, or by stress tests with Random Query Generator.
2021-01-29 18:03:20 +02:00
Vladislav Vaintroub
d8373fea5f MDEV-24685 - remove IO thread states output from SHOW ENGINE INNODB STATUS
There are no IO threads anymore.
2021-01-29 18:02:14 +02:00
Sergei Petrunia
c36720388d MDEV-9750: Quick memory exhaustion with 'extended_keys=on' ...
(Variant , full patch, for 10.5)

Do not produce SEL_ARG graphs that would yield huge numbers of ranges.
Introduce a concept of SEL_ARG graph's "weight". If we are about to
produce a graph whose "weight" exceeds the limit, remove the parts
of SEL_ARG graph that represent the biggest key parts. Do so until
the graph's is within the limit.

Includes
- debug code to verify SEL_ARG graph weight
- A user-visible @@optimizer_max_sel_arg_weight to control the optimization
- Logging the optimization into the optimizer trace.
2021-01-29 16:20:57 +03:00
sjaakola
a2eb974b50 MDEV-24721 galera.mysql-wsrep-bugs-607 test failure
The implementation for MDEV-17048 apperas to be direct copy from mysql version.
The group commit works differently in mariadb and the assert in wsrep_unregister_from_group_commit() is too strict.

The reason is that in: Wsrep_high_priority_service::log_dummy_write_set(), the transaction will undergo full rollback:
    {
      cs.before_rollback();
      cs.after_rollback();
    }

After that, the client's transaction state is set to be:  wsrep::transaction::s_aborted.
The execution then continues execution by:

...
 wsrep_register_for_group_commit(m_thd);
...
 wsrep_unregister_from_group_commit(m_thd);

The bogus assert in wsrep_unregister_from_group_commit() allows only transactions states of :s_ordered_commit or s_aborting.

As the fix, I brought back the same assert as is present in MariaDB 10.4 version.
2021-01-29 12:14:08 +02:00
Anel Husakovic
85130c5a4f MDEV-24093: Detect during mysql_upgrade if type_mysql_json.so is needed and load it
a. The change makes `mariadb-upgrade` detect if `MYSQL_JSON` data type is needed.
b. Install the data type if it's not installed.
c. Uninstalls the data type once finished.
d. Create `.opt` and `.inc` files `have_type_mysql_json` and adapt the
tests

Reviewed by: vicentiu@mariadb.org
2021-01-28 19:01:32 +01:00
Marko Mäkelä
6d1f1b61b5 MDEV-24564 Statistics are lost after ALTER TABLE
Ever since commit 007f68c37f,
ALTER TABLE no longer invokes handler::open() after
handler::commit_inplace_alter_table().

ha_innobase::reload_statistics(): Reload or recompute statistics
after ALTER TABLE.

innodb_notify_tabledef_changed(): A new function to invoke
ha_innobase::reload_statistics().

handlerton::notify_tabledef_changed(): Add the parameter handler*
so that ha_innobase::reload_statistics() can be invoked.

ha_partition::notify_tabledef_changed(),
partition_notify_tabledef_changed(): Pass through the call
to any partitions or subpartitions.

This is based on code that was supplied by Monty.
2021-01-28 14:15:01 +02:00
Vlad Lesin
744e9752d8 MDEV-24705 add check that LSN of the last skipped log record equals to FIL_PAGE_LSN field
The check is added.
2021-01-28 14:31:25 +03:00
Thirunarayanan Balathandayuthapani
6e80a34d68 MDEV-24695 Encryption modifies a freed page
During recovery, InnoDB fails if it tries to apply a FREE_PAGE
and WRITE record to the page. InnoDB encryption thread accesses
the freed page and writes redo log for it.

This is similar to commit deadec4e68 (MDEV-24569)
InnoDB is missing buf_page_free() while freeing the segment.
To avoid accessing of freed page in buffer pool, InnoDB should
mark the pages as FREED while freeing the segment. Also to
avoid reading of freed page, InnoDB should check the
allocation bitmap page.

fseg_free_step(): Mark the page in buffer pool as FREED

fseg_free_step_not_header(): Mark the page in buffer pool as FREED

buf_dump(): Ignore the freed pages while dumping the buffer pool content

fil_crypt_get_page_throttle_func(): Skip the rotation for FREED page
to avoid the assert failure during recovery

fil_crypt_rotate_page(): Skip the rotation for the freed page

Reviewed-by: Marko Mäkelä
2021-01-28 13:26:34 +05:30
Marko Mäkelä
c6308355e5 MDEV-24612 fixup: Skip the test for --embedded 2021-01-28 07:51:43 +02:00
Thirunarayanan Balathandayuthapani
700ae20d45 MDEV-24693 LeakSanitizer: detected memory leaks in mem_heap_create_block_func / fts_optimize_create_msg
- This issue is caused by the commit bf1f9b59c7
(MDEV-24638). Delay the creation of SYNC message in
fts_optimize_request_sync_table. So that InnoDB can avoid creating
the message if the table already has SYNC message in fts_optimize_wq queue
2021-01-27 20:41:36 +05:30
Jan Lindström
30379d906b Update disabled.def in suites
* galera
* galera_sr
* galera_3nodes
2021-01-27 14:16:47 +02:00
Jan Lindström
e2bcf68279 MDEV-24010 : galera_3nodes.GCF-354 MTR fails : WSREP has not yet prepared node for application use
Correct test.
2021-01-27 10:55:10 +02:00
Roman Nozdrin
0565d19973 MDEV-24298 Select Handler now process INSERT..SELECT with a single derived at
the top level
2021-01-26 14:07:14 +00:00
Marko Mäkelä
927a882341 Merge 10.4 into 10.5 2021-01-25 15:06:52 +02:00
Marko Mäkelä
e626f511f9 MDEV-24653 fixup: Make the test deterministic 2021-01-25 14:56:38 +02:00
Marko Mäkelä
5db3827689 Merge 10.3 into 10.4 2021-01-25 14:43:07 +02:00
Marko Mäkelä
75538f94ca MDEV-24653 fixup: Make the test deterministic 2021-01-25 14:40:22 +02:00
Marko Mäkelä
0c3d264207 instant_alter_debug: Cover everything with innodb_instant_alter_column 2021-01-25 13:56:10 +02:00
Marko Mäkelä
961c7938bb Merge 10.4 into 10.5 2021-01-25 12:44:24 +02:00
Marko Mäkelä
3467f63764 Merge 10.3 into 10.4 2021-01-25 11:02:07 +02:00
Marko Mäkelä
eaeb8ec4b8 MDEV-24653 Assertion block->page.id.page_no() == index->page failed in innobase_add_instant_try()
We may end up with an empty leaf page (containing only an ADD COLUMN
metadata record) that is not the root page.

innobase_add_instant_try(): Disable an optimization for a non-canonical
empty table that contains a metadata record somewhere else than in
the root page.

btr_pcur_store_position(): Tolerate a non-canonical empty table.
2021-01-25 10:24:35 +02:00
Marko Mäkelä
5adcb2e7b8 MDEV-24661: Disable an unstable test 2021-01-23 17:56:50 +02:00
Marko Mäkelä
84b8f529c7 MDEV-24659 Assertion !fsp_is_system_temporary(bpage->id().space()) failed in buf_flush_relocate_on_flush_list()
When commit 5eb539555b (MDEV-12227)
removed the pages of temporary tables from the buf_pool.flush_list,
an adjustment to the buffer pool resizing was forgotten.

buf_pool_t::realloc(): Do not invoke buf_flush_relocate_on_flush_list()
for pages that belong to the temporary tablespace. Also, deduplicate
some code at the end.

buf_page_t::set_corrupt_id(): Tolerate oldest_modification()==1
(the dummy value) for temporary tablespace pages. The revised
buf_pool_t::realloc() may invoke this on dirty temporary tablespace pages.
2021-01-23 17:45:03 +02:00
Marko Mäkelä
0e10d7ea14 MDEV-22351 InnoDB may recover wrong information after RESET MASTER
Ever since commit 947efe17ed
InnoDB no longer writes binlog position in one place.
It will not at all be written to the TRX_SYS page, and
instead it will be written to the undo log header page that
changes the transaction state.

trx_rseg_mem_restore(): Recover the information from the latest
written page.
2021-01-22 16:44:17 +02:00
Thirunarayanan Balathandayuthapani
bf1f9b59c7 MDEV-24638 Avoid repetitive FTS SYNC request for table
fts_optimize_request_sync_table() can avoid the repetitive
FTS SYNC request of the table if the table already has FTS_SYNC
message in fts_optimize_wq queue.

Reviewed-by: Marko Mäkelä
2021-01-22 18:27:08 +05:30
Jan Lindström
ce141d0714 MDEV-24463 : galera.galera_sst_mysqldump_with_key MTR failed: 'INSERT failed: 1213: Deadlock found when trying to get lock
We need to complete SST if both new and old start positions are
not same as initial positions. If they are initial positions
just set local uuid and seqno.
2021-01-22 12:12:42 +02:00
Thirunarayanan Balathandayuthapani
816808d6c9 MDEV-24652 mtr fails while reusing the cached undo log block
While reusing the cached undo log block, mtr expects the page
write to change while writing the trx id. cached undo log block
could contain bytes which were originally written for some other
transaction. So InnoDB should make mtr to do MAYBE_NOP while reusing
cached undo log block.

Reviewed-by: Marko Mäkelä
2021-01-22 14:58:49 +05:30
Sergei Golubchik
4e503aec7f MDEV-24593 Signal 11 when group by primary key of table joined to information_schema.columns
I_S tables were materialized too late, an attempt to use table
statistics before the table was created caused a crash.

Let's move table creation up. it only needs read_set to
be calculated properly, this happens in JOIN::optimize_inner(),
after semijoin transformation.

Note that tables are not populated at that point, so most of the
statistics would make no sense anyway. But at least field sizes
will be correct. And it won't crash.
2021-01-22 00:04:31 +01:00