Commit graph

24215 commits

Author SHA1 Message Date
Nayuta Yanagisawa
2b89598fe0 MDEV-26852 Spider: -Werror=maybe-uninitialized raises on spd_sys_table.cc and ha_spider.cc
The warning "maybe-uninitialized" raises on spd_sys_table.cc and
ha_spider.cc when the sever is compiled with CMAKE_CXX_FLAGS=-Og.
2022-09-26 23:23:13 +09:00
Andrew Hutchings
59c9e2f202
MDEV-29579 Fix CONNECT ASAN hits (#2277)
There are currently two things causing ASAN hits on CONNECT engine when
the plugin is used as a dynamic module. These are libxml2 and libodbc.

libxml2 has some quirks when not the first and last thing called in the
main thread of an application, some of the global memory isn't cleaned
up correctly. The same is assumed of libodbc but this does not have
explicit API for this.

This is being fixed in two ways.

First we are removing the libxml2 cleanup call. This is because the
current one is messy and whatever it fixed has gone away. But also
because if this is called and libxml2 is used again this can cause
issues. For example if two different plugins to MariaDB both happen to
use libxml2.

The second fix is a hack that exploits `STB_GNU_UNIQUE` so that when
compiled with ASAN the plugin will remain in memory after dlclose().
This allows libodbc to cleanup and has the added advatage that we will
get clean stacks from ASAN for CONNECT when the leak is detected at the
end of execution.

Details of the `STB_GNU_UNIQUE` method can be found here:

https://web.archive.org/web/20100730094324/http://www.redhat.com/archives/posix-c++-wg/2009-August/msg00002.html
2022-09-26 13:59:23 +01:00
Marko Mäkelä
e3fdabd501 MDEV-29613 fixup: clang -Wunused-but-set-variable 2022-09-26 15:16:51 +03:00
Anel
b3e06ce3fd
MDEV-28533: CONNECT engine does not quote columns involved in WHERE clause (#2263) 2022-09-26 07:53:43 +01:00
Marko Mäkelä
3c92050d1c Fix build without either ENABLED_DEBUG_SYNC or DBUG_OFF
There are separate flags DBUG_OFF for disabling the DBUG facility
and ENABLED_DEBUG_SYNC for enabling the DEBUG_SYNC facility.
Let us allow debug builds without DEBUG_SYNC.

Note: For CMAKE_BUILD_TYPE=Debug, CMakeLists.txt will continue to
define ENABLED_DEBUG_SYNC.
2022-09-23 17:37:52 +03:00
Andrew Hutchings
66cd1c33f7
MDEV-25767 Fix CONNECT ODBC WHERE condition crash (#2243)
When an UPDATE or DELETE was executed with a WHERE condition it would
crash the MariaDB server. This is because the code expects the WHERE
condition to have generated a query string but it hasn't.

Also updates the ODBC test results for current MariaDB version.
2022-09-23 14:02:41 +01:00
Marko Mäkelä
13eae1885e Merge 10.3 into 10.4 2022-09-23 13:47:15 +03:00
Marko Mäkelä
a69cf6f07e MDEV-29613 Improve WITH_DBUG_TRACE=OFF
In commit 28325b0863
a compile-time option was introduced to disable the macros
DBUG_ENTER and DBUG_RETURN or DBUG_VOID_RETURN.

The parameter name WITH_DBUG_TRACE would hint that it also
covers DBUG_PRINT statements. Let us do that: WITH_DBUG_TRACE=OFF
shall disable DBUG_PRINT() as well.

A few InnoDB recovery tests used to check that some output from
DBUG_PRINT("ib_log", ...) is present. We can live without those checks.

Reviewed by: Vladislav Vaintroub
2022-09-23 13:40:42 +03:00
Marko Mäkelä
ce23802c0e MDEV-29600 Memory leak in row_log_table_apply_update()
row_log_table_apply_update(): Free the pcur.old_rec_buf before returning.
It may be allocated by btr_pcur_store_position() inside
btr_blob_log_check_t::check() and btr_store_big_rec_extern_fields().

This memory leak was introduced in
commit 2e814d4702 (MariaDB Server 10.2.2)
via mysql/mysql-server@ce0a1e85e2
(MySQL 5.7.5).
2022-09-22 11:18:00 +03:00
Alexey Botchkov
f5e4e15403 Backport fix for MDEV-29352 to 10.3-10.5
The fix for MDEV-29352 was pushed to 10.6+ but the code causing the
bug is old and the bug is unlikely to be a recent regression in 10.6.
So, we apply the fix also to older versions, 10.3-10.5.

The original commit message:

MDEV-29352 SIGSEGV's in strlen and unknown location on optimized builds at SHUTDOWN

When the UDF creation frails to write the newly created UDF into
the related system table, the UDF is still created in memory.

However, as it is now, the related DLL is unloaded in this case right
in the mysql_create_function. And failure happens when the UDF handle
is freed and tries to unload the respective DLL which is still unloaded.
2022-09-21 22:02:21 +09:00
Sergei Golubchik
fc8a765532 MDEV-29480 spider group by handler wrong result on order by aggregate alias
when generating a query to send to a remote server, spider generates
new aliases for all tables in the query (at least in the group_by handler).
First it walks all the expressions and create a list of new table aliases
to use for each field. Then - in init_scan() - it actually generates the
query, taking for each field the next alias from the list.

It dives recursively into functions, for example, for func(f1) it'll
go in, will see the field f1 and append to the list the new name for
the table of f1. This works fine for non-aggregate functions and
for aggregate functions in the SELECT list. But aggregate functions
in the ORDER BY are always references to the select list, they never
need to be qualified with a table name. That is, even if there is a
field name as an argument of an aggregate function in the ORDER BY
it must not append a table alias to the list. Let's just skip
aggregate functions when analyzing ORDER BY for table aliases.

This fixes spider/bugfix.mdev_29008
(was observed on aarch64, x86, ppc64le, and amd64 --rr)
2022-09-20 22:59:13 +09:00
Marko Mäkelä
0c0a569028 Merge 10.3 into 10.4 2022-09-20 12:38:25 +03:00
Andrew Hutchings
65b4a2afb8
MDEV-29426 Fix memory leak in CONNECT JSON/BSON (#2255)
* MDEV-29426 Fix memory leak in CONNECT JSON/BSON

If information is being gathered on JSON/BSON tables we leak the heap
allocated to the table class. With this fix we close and cleanup
instead, just as we do for XML tables.
2022-09-19 13:25:45 +01:00
Marko Mäkelä
5e959bc363 Fix clang -Wunused-but-set-variable 2022-09-19 13:30:52 +03:00
Marko Mäkelä
3ab4b260e1 Merge innodb.cmake to CMakeLists.txt
The reason why mysql/mysql-server@8020cfac20
split the files was some unit tests that never existed in the
MariaDB Server code base. The storage/innobase/unittest/ works just fine
with this file.

This is reverting part of 2e814d4702
which applied InnoDB changes from MySQL 5.7.9.
2022-09-19 12:42:50 +03:00
Marko Mäkelä
73658eded3 Cleanup: Remove HAVE_IB_LINUX_FUTEX
The futex system calls were introduced in Linux 2.6.0,
which was released in December 2003. It should be safe to assume
that the system calls are always available on the Linux kernels
that MariaDB Server 10.3 would run on.
2022-09-19 12:36:19 +03:00
Marko Mäkelä
4c8b65db08 Cleanup: Remove INNODB_COMPILER_HINTS
There should be no point to disable branch prediction hints or prefetch.
2022-09-19 12:29:16 +03:00
Marko Mäkelä
c22dff21a5 InnoDB cleanup: Replace UNIV_LINUX, UNIV_SOLARIS, UNIV_AIX
Let us use the normal platform-specific preprocessor symbols
__linux__, __sun__, _AIX instead of some homebrew ones.

The preprocessor symbol UNIV_HPUX must have lost its meaning
by f6deb00a56 (note: the symbol
UNIV_HPUX10 is being checked for, but only UNIV_HPUX is defined).
2022-09-19 12:20:53 +03:00
Marko Mäkelä
bbf81b51f2 Correct typos in a function comment
Thanks to Thirunarayanan Balathandayuthapani for spotting this.
2022-09-19 10:23:57 +03:00
Thirunarayanan Balathandayuthapani
d7aefc0fab MDEV-29479 I_S.INNODB_SYS_TABLESPACES doesn't have temporary tablespace information
- innodb_sys_tablespaces view in information schema displays temporary
tablespace information too.
2022-09-14 13:37:14 +05:30
Marko Mäkelä
18795f5512 Merge 10.3 into 10.4 2022-09-13 16:36:38 +03:00
Marko Mäkelä
68ce0231ad MDEV-23801 Assertion failed in btr_pcur_store_position()
btr_lift_page_up(): If the leaf page only contains a hidden metadata
record for MDEV-11369 instant ADD COLUMN, convert the table to the
canonical format like we are supposed to do whenever the table
becomes empty.
2022-09-13 15:46:40 +03:00
Alexander Barkov
4c14243373 A cleanup for MDEV-29446 Change SHOW CREATE TABLE to display default collation
Recording test results according to MDEV-29446 changes:

storage/rocksdb/mysql-test/rocksdb/r/use_direct_io_for_flush_and_compaction.result
2022-09-13 12:44:23 +04:00
Marko Mäkelä
fc794fd8ff MDEV-29520 heap-use-after-poison in row_merge_spatial_rows()
row_merge_read_clustered_index(): Do not call mem_heap_empty(row_heap)
before row_merge_spatial_rows() has been able to read the data.
2022-09-13 08:58:34 +03:00
Alexander Barkov
f1544424de MDEV-29446 Change SHOW CREATE TABLE to display default collation 2022-09-12 22:10:39 +04:00
Marko Mäkelä
667df98c3e MDEV-29507 InnoDB: Failing assertion: table->n_rec_locks == 0
lock_place_prdt_page_lock(): Do not place locks on temporary tables.
Temporary tables can only be accessed from one connection, so
it does not make any sense to acquire any transactional locks on them.
2022-09-12 09:27:46 +03:00
Marko Mäkelä
c7ba237793 Merge 10.3 into 10.4 2022-09-07 08:08:59 +03:00
Thirunarayanan Balathandayuthapani
ac49b7a845 MDEV-29342 Assertion failure in file que0que.cc line 728
- During shutdown, InnoDB fts fails to update synced doc id
when there is only one doc id about to sync. While starting
the server, InnoDB fetches the already synced doc id from
config table. In the subsequent sync operation, InnoDB fails
with DB_DUPLICATE_KEY error.
2022-09-06 17:23:31 +05:30
Nayuta Yanagisawa
e4cffc92c7 MDEV-27172 Prefix indices on Spider tables may lead to wrong query results
Spider converts HA_READ_KEY_EXACT to the equality (=) in the
function spider_db_append_key_where_internal() but the conversion
is not necessarily correct for tables with prefix indices.

We fix the bug by converting HA_READ_KEY_EXACT to 'LIKE "foo%"' when
a target key is a prefix key. The fix is partly inspired by FEDERATED.
See ha_federated::create_where_from_key() for more details.
2022-09-01 18:32:40 +09:00
Daniele Sciascia
2917bd0d2c Reduce compilation dependencies on wsrep_mysqld.h
Making changes to wsrep_mysqld.h causes large parts of server code to
be recompiled. The reason is that wsrep_mysqld.h is included by
sql_class.h, even tough very little of wsrep_mysqld.h is needed in
sql_class.h. This commit introduces a new header file, wsrep_on.h,
which is meant to be included from sql_class.h, and contains only
macros and variable declarations used to determine whether wsrep is
enabled.
Also, header wsrep.h should only contain definitions that are also
used outside of sql/. Therefore, move WSREP_TO_ISOLATION* and
WSREP_SYNC_WAIT macros to wsrep_mysqld.h.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
2022-08-31 11:05:23 +03:00
Marko Mäkelä
7e574eb52c Merge 10.3 into 10.4 2022-08-30 12:17:33 +03:00
Marko Mäkelä
422f3204ef MDEV-29409 Buffer overflow in my_wc_mb_filename() on RENAME TABLE
dict_table_rename_in_cache(), dict_table_get_highest_foreign_id():
Reserve sufficient space for the fkid[] buffer, and ensure that the
fkid[] will be NUL-terminated.

The fkid[] must accommodate both the database name (which is already
encoded in my_charset_filename) and the constraint name
(which must be converted to my_charset_filename) so that we can check
if it is in the format databasename/tablename_ibfk_1 (all encoded in
my_charset_filename).
2022-08-30 12:02:56 +03:00
Marko Mäkelä
b260903832 MDEV-29258 Failing assertion for name length on RENAME TABLE
trx_undo_page_report_rename(): Use the correct maximum length of
a table name. Both the database name and the table name can be up to
NAME_CHAR_LEN (64 characters) times 5 bytes per character in the
my_charset_filename encoding. They are not encoded in UTF-8!

fil_op_write_log(): Reserve the correct amount of log buffer for
a rename operation. The file name will be appended by
mlog_catenate_string().

rename_file_ext(): Reserve a large enough buffer for the file names.
2022-08-30 10:59:31 +03:00
Marko Mäkelä
851058a3e6 Merge 10.3 into 10.4 2022-08-25 15:17:20 +03:00
Marko Mäkelä
d1a80c42ee MDEV-29384 Hangs caused by innodb_adaptive_hash_index=ON
buf_defer_drop_ahi(): Remove. Ever since
commit c7f8cfc9e7 (MDEV-27700)
it is safe to invoke btr_search_drop_page_hash_index(block, true)
to remove an orphan adaptive hash index.

Any attempt to upgrade page latches is prone to deadlocks. Recently,
we observed a few hangs that involved nothing more than a small table
consisting of one clustered index page, one secondary index page and
some undo pages.
2022-08-25 15:14:38 +03:00
Marko Mäkelä
f2a53b6158 btr_search_drop_page_hash_index(): Remove a racey debug check 2022-08-24 15:00:47 +03:00
Thirunarayanan Balathandayuthapani
61f456e772 MDEV-29319 Assertion failure size_in_header >= space.free_limit in fsp_get_available_space_in_free_extents()
- Don't remove the constant parameter in fsp_get_available_space_in_free_extents()
2022-08-24 12:27:15 +05:30
Thirunarayanan Balathandayuthapani
dd737d071e MDEV-29291 Assertion `!table->fts' failed in dict_table_can_be_evicted on SHUTDOWN
- InnoDB fts table initially added to LRU table cache
while creating the table. Later, table was marked
as non-evicted when we add the table to fts optimizer
list. Before marking the table as non-evicted, master
thread can try to evict the fts table.
2022-08-24 10:58:08 +05:30
Thirunarayanan Balathandayuthapani
0b80573310 MDEV-29319 Assertion failure size_in_header >= space.free_limit in fsp_get_available_space_in_free_extents()
- Race condition between fsp_get_available_space_in_free_extents()
and fsp_try_extend_data_file() while accessing space.free_limit.
Before calling fsp_get_available_space_in_free_extents(), take
shared lock on space->latch.
2022-08-24 10:58:08 +05:30
Nayuta Yanagisawa
e404315258 Fix wrong diff introduced by merge commit
Many Spider tests were broken by the merge commit, 36d173e.
2022-08-23 19:59:30 +09:00
Marko Mäkelä
b68ae6dc1d Merge 10.3 into 10.4 2022-08-22 16:22:09 +03:00
Thirunarayanan Balathandayuthapani
c7f8cfc9e7 MDEV-27700 ASAN: Heap_use_after_free in btr_search_drop_page_hash_index()
Reason:
=======
Race condition between btr_search_drop_hash_index() and
btr_search_lazy_free(). One thread does resizing of buffer pool
and clears the ahi on all pages in the buffer pool, frees the
index and table while removing the last reference. At the same time,
other thread access index->heap in btr_search_drop_hash_index().

Solution:
=========
Acquire the respective ahi latch before checking index->freed()

btr_search_drop_page_hash_index(): Added new parameter to indicate
that drop ahi entries only if the index is marked as freed

btr_search_check_marked_free_index(): Acquire all ahi latches and
return true if the index was freed
2022-08-22 16:29:46 +05:30
Marko Mäkelä
36d173e523 Merge 10.3 into 10.4 2022-08-22 12:34:42 +03:00
Thirunarayanan Balathandayuthapani
32167225c7 MDEV-13013 InnoDB unnecessarily extends data files
- While creating a new InnoDB segment, allocates the extent
before allocating the inode or page allocation even though
the pages are present in fragment segment. This patch does
reserve the extent when InnoDB ran out of fragment pages
in the tablespace.
2022-08-17 11:08:49 +05:30
Nayuta Yanagisawa
c208006080 MDEV-29008 Server crash or assertion `field' failed in spider_db_open_item_ident / group by handler
ha_spider::field_exchange() returns NULL and that results in a crash
or a assertion failure in spider_db_open_item_ident().

In the first place, there seems to be no need to call field_exchange()
for printing an identity (column name with alias). Thus, we simply
remove the call.
2022-08-16 16:36:44 +09:00
Oleksandr Byelkin
e509065247 Merge branch '10.3' into 10.4 2022-08-03 19:51:44 +02:00
Thirunarayanan Balathandayuthapani
f9ec9b6abb MDEV-27282 InnoDB: Failing assertion: !query->intersection
- query->intersection fails to get freed if the query exceeds
innodb_ft_result_cache_limit

- errors from init_ftfuncs were not propogated by delete command

This is taken from percona/percona-server@ef2c0bcb9a
2022-08-03 20:35:12 +05:30
Oleksandr Byelkin
48e35b8cf6 Merge branch '10.3' into 10.4 2022-08-02 14:15:39 +02:00
Oleksandr Byelkin
5ac528a91f Merge remote-tracking branch 'connect/10.3' into 10.3 2022-08-02 10:23:53 +02:00
Daniel Black
182a6383cd MDEV-16605 Always include buf_madvise_do_dump in binaries
The "used" attribute seems to do this

ref: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#Common-Function-Attributes
2022-08-02 12:29:11 +10:00