Commit graph

69 commits

Author SHA1 Message Date
Thirunarayanan Balathandayuthapani
f9ec9b6abb MDEV-27282 InnoDB: Failing assertion: !query->intersection
- query->intersection fails to get freed if the query exceeds
innodb_ft_result_cache_limit

- errors from init_ftfuncs were not propogated by delete command

This is taken from percona/percona-server@ef2c0bcb9a
2022-08-03 20:35:12 +05:30
Marko Mäkelä
e4a7c15dd6 Merge 10.2 into 10.3 2021-10-21 13:41:04 +03:00
Thirunarayanan Balathandayuthapani
8ce8c269f4 MDEV-19522 InnoDB commit fails when FTS_DOC_ID value is greater than 4294967295
InnoDB commit fails when consecutive FTS_DOC_ID value
is greater than 4294967295.
Fix is that InnoDB should remove the delta FTS_DOC_ID
value limitations and fts should encode 8 byte value,
remove FTS_DOC_ID_MAX_STEP variable. Replaced the
fts0vlc.ic file with fts0vlc.h

fts_encode_int(): Should be able to encode 10 bytes value

fts_get_encoded_len(): Should get the length of the value
which has 10 bytes

fts_decode_vlc(): Add debug assertion to verify the maximum
length allowed is 10.

mach_read_uint64_little_endian(): Reads 64 bit stored in
little endian format

Added a unit test case which check for minimum and maximum
value to do the fts encoding
2021-10-21 12:56:59 +03:00
Marko Mäkelä
cd65845a0e Merge 10.2 into 10.3
MDEV-18734 FIXME: vcol.partition triggers ASAN heap-use-after-free
2021-08-18 12:26:58 +03:00
Marko Mäkelä
4cd063b9e4 MDEV-26376 pars_info_bind_id() unnecessarily copies strings
pars_info_bind_id(): Remove the parameter copy_name. It was always
being passed as constant TRUE or true. It turns out that copying
the string is completely unnecessary. In all calls except the one
in fts_get_select_columns_str() and fts_doc_fetch_by_doc_id(),
the parameter is being passed as a compile-time constant, and therefore
the pointer cannot become stale. In that special call, the string
that is being passed is allocated from the same memory heap that
pars_info_bind_id() would have been using.

pars_info_add_id(): Remove (unused declaration).
2021-08-16 12:10:20 +03:00
Marko Mäkelä
a8de8f261d Merge 10.2 into 10.3 2020-10-28 10:01:50 +02:00
Sachin Agarwal
e391417f0f Bug INNODB FTS PHRASE SEARCH HIT AN ASSERT
Problem:
In Full-text phrase search, we filter out row that do not contain
all the tokens in the phrase.
If we do not filter out doc_id that doesn't appear in all the
token's doc_id lists then we hit an assert.

Fix:
if any of the token has last doc_id equal to ith doc_id of the first
token doc_id list then filter out rest of the higher doc_ids.

RB: 24909
Reviewed by : Annamalai Gurusami <annamalai.gurusami@oracle.com>

This is a cherry-pick of
mysql/mysql-server@5aa075277d
but without a test case, because the test case depends on an n-gram
tokenizer that will be missing from MariaDB until MDEV-10267 is added.
2020-10-26 12:21:29 +02:00
Marko Mäkelä
e3d692aa09 Merge 10.2 into 10.3 2020-10-22 08:26:28 +03:00
Marko Mäkelä
620ea816ad Merge 10.1 into 10.2 2020-10-21 14:02:04 +03:00
Marko Mäkelä
c7552969d0 MDEV-23999 Potential stack overflow in InnoDB fulltext search
fts_query_t::nested_sub_exp: Keep track of nested
fts_ast_visit_sub_exp() calls.

fts_ast_visit_sub_exp(): Return DB_OUT_OF_MEMORY if the
maximum recursion depth is exceeded.

This is motivated by a change in MySQL 5.6.50:
mysql/mysql-server@e2a46b4834
Bug  USING MANY NESTED ARGUMENTS WITH BOOLEAN FTS CAN LEAD
TO TERMINATE SERVER
2020-10-21 10:04:44 +03:00
Marko Mäkelä
c277bcd591 Merge 10.2 into 10.3 2020-08-21 19:18:34 +03:00
Marko Mäkelä
4bd56a697f Merge 10.2 into 10.3 2020-08-13 18:18:25 +03:00
Marko Mäkelä
31aef3ae99 Fix GCC 10.2.0 -Og -Wmaybe-uninitialized
For some reason, GCC emits more -Wmaybe-uninitialized warnings
when using the flag -Og than when using -O2. Many of the warnings
look genuine.
2020-08-11 15:58:16 +03:00
Marko Mäkelä
680463a8d9 Merge 10.2 into 10.3 2020-06-05 16:51:26 +03:00
Marko Mäkelä
eba2d10ac5 MDEV-22721 Remove bloat caused by InnoDB logger class
Introduce a new ATTRIBUTE_NOINLINE to
ib::logger member functions, and add UNIV_UNLIKELY hints to callers.

Also, remove some crash reporting output. If needed, the
information will be available using debugging tools.

Furthermore, remove some fts_enable_diag_print output that included
indexed words in raw form. The code seemed to assume that words are
NUL-terminated byte strings. It is not clear whether a NUL terminator
is always guaranteed to be present. Also, UCS2 or UTF-16 strings would
typically contain many NUL bytes.
2020-06-04 10:24:10 +03:00
Marko Mäkelä
f4f0ef3e37 Merge 10.2 into 10.3 2020-05-20 11:41:51 +03:00
Yury Kurlykov
d2900d917f
MDEV-22629 Remove fts_indexes field from struct fts_update_t ()
fts_indexes field in fts_update_t never used. So replace fts_update_t
with doc_id_t in the code
2020-05-20 12:40:07 +05:30
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Oleksandr Byelkin
c51f85f882 Merge branch '10.2' into 10.3 2019-05-12 17:20:23 +02:00
Vicențiu Ciorbaru
c0ac0b8860 Update FSF address 2019-05-11 19:25:02 +03:00
Marko Mäkelä
b2f3755c8e Merge 10.1 into 10.2 2019-05-10 08:02:21 +03:00
Marko Mäkelä
f3718a112a MDEV-18220: Backport some code from MariaDB 10.2
fts_get_table_name(): Output to a caller-allocated buffer.

fts_get_table_name_prefix(): Use the lower-overhead allocation
ut_malloc() instead of mem_alloc().

This is based on mysql/mysql-server@d1584b9f38
in MySQL 5.7.4.
2019-05-10 07:57:01 +03:00
Marko Mäkelä
f92749ed36 MDEV-18220: heap-use-after-free in fts_get_table_name_prefix()
fts_table_t::parent: Remove the redundant field. Refer to
table->name.m_name instead.

fts_update_sync_doc_id(), fts_update_next_doc_id(): Remove
the redundant parameter table_name.

fts_get_table_name_prefix(): Access the dict_table_t::name.
FIXME: Ensure that this access is always covered by
dict_sys->mutex.
2019-05-10 07:57:01 +03:00
Marko Mäkelä
fd58bb71e2 Merge 10.2 into 10.3 2018-11-19 18:45:53 +02:00
Marko Mäkelä
ff88e4bb8a Remove many redundant #include from InnoDB 2018-11-19 11:42:14 +02:00
Marko Mäkelä
df563e0c03 Merge 10.2 into 10.3
main.derived_cond_pushdown: Move all 10.3 tests to the end,
trim trailing white space, and add an "End of 10.3 tests" marker.
Add --sorted_result to tests where the ordering is not deterministic.

main.win_percentile: Add --sorted_result to tests where the
ordering is no longer deterministic.
2018-11-06 09:40:39 +02:00
Marko Mäkelä
32062cc61c Merge 10.1 into 10.2 2018-11-06 08:41:48 +02:00
Thirunarayanan Balathandayuthapani
1dacd5f299 MDEV-12547: InnoDB FULLTEXT index has too strict innodb_ft_result_cache_limit max limit
- Backported the MYSQL_SYSVAR_SIZE_T to 10.0
- The parameter innodb_ft_result_cache_limit was only 32 bits wide
also on 64-bit systems. Make it size_t, so that it will be 64 bits
on 64-bit systems.
- Added a test case that show how innodb_ft_result_cache_limit variables
behaves in 32bit and 64 bit system.
2018-10-16 13:02:50 +05:30
Marko Mäkelä
05459706f2 Merge 10.2 into 10.3 2018-08-03 15:57:23 +03:00
Marko Mäkelä
814ae57daf Merge 10.1 into 10.2 2018-08-03 13:02:56 +03:00
Marko Mäkelä
b963cbaf4b Follow-up fix to MDEV-16865: InnoDB fts_query() ignores KILL
fts_query(): Remove a redundant condition (result will never be NULL),
and instead check if *result is NULL, to prevent SIGSEGV in
fts_query_free_result().
2018-08-03 11:49:49 +03:00
Marko Mäkelä
ef3070e997 Merge 10.1 into 10.2 2018-08-02 08:19:57 +03:00
Marko Mäkelä
a7f84f09bf MDEV-16865 InnoDB fts_query() ignores KILL
The functions fts_ast_visit() and fts_query() inside
InnoDB FULLTEXT INDEX query processing are not checking
for THD::killed (trx_is_interrupted()), like anything
that potentially takes a long time should do.

This is a port of the following change from MySQL 5.7.23,
with a completely rewritten test case.

commit c58c6f8f66ddd0357ecd0c99646aa6bf1dae49c8
Author: Aakanksha Verma <aakanksha.verma@oracle.com>
Date:   Fri May 4 15:53:13 2018 +0530

Bug  MAX_EXECUTION_TIME NOT INTERUPTED WITH FULLTEXT SEARCH USING MECAB
2018-08-01 08:43:12 +03:00
Marko Mäkelä
2b27ac8282 Fix many -Wunused-parameter
Remove unused InnoDB function parameters and functions.

i_s_sys_virtual_fill_table(): Do not allocate heap memory.

mtr_is_block_fix(): Replace with mtr_memo_contains().

mtr_is_page_fix(): Replace with mtr_memo_contains_page().
2018-05-01 16:52:19 +03:00
Marko Mäkelä
ba19764209 Fix most -Wsign-conversion in InnoDB
Change innodb_buffer_pool_size, innodb_fill_factor to unsigned.
2018-04-28 20:45:45 +03:00
Sergey Vojtovich
3d5f7ad23a MDEV-15773 - Simplified away trx_free_for_(mysql|background) 2018-04-04 14:09:37 +04:00
Sergey Vojtovich
d6d58836bb MDEV-15773 - trx_allocate_for_background() -> trx_create()
trx_free_resurrected(): Remove, unused function
2018-04-04 14:09:37 +04:00
Sergei Golubchik
4771ae4b22 Merge branch 'github/10.1' into 10.2 2018-02-06 14:50:50 +01:00
Marko Mäkelä
706ed8552d Revert "MDEV-6928: Add trx pointer to struct mtr_t"
This reverts commit 3486135bb5.

The commit comment ended in the words: "This is needed later."
Apparently the "later" never arrived.
2018-01-29 11:05:17 +02:00
Vicențiu Ciorbaru
3699a4b5c0 Merge branch 'merge-innodb-5.6' into 10.0 2018-01-24 18:23:25 +02:00
Vicențiu Ciorbaru
3dfe148074 5.6.39 2018-01-23 17:43:37 +02:00
Marko Mäkelä
786a722eba Merge 10.1 into 10.2 2017-10-27 10:24:02 +03:00
Vicențiu Ciorbaru
3b35d745c3 Merge branch 'merge-innodb-5.6' into 10.0 2017-10-26 12:46:47 +03:00
Vicențiu Ciorbaru
4ef64e01a7 5.6.38 2017-10-25 21:35:33 +03:00
Sergei Golubchik
e0a1c745ec Merge branch '10.1' into 10.2 2017-10-24 14:53:18 +02:00
Marko Mäkelä
babbf8c6fc fts_query_free(): Fix a potential assertion failure
The ownership of the field query->intersection usually transfers
to query->doc_ids. In some error scenario, it could be possible
that fts_query_free() would be invoked with query->intersection!=NULL.
Let us handle that case, instead of intentionally crashing the server.
2017-10-18 10:22:10 +03:00
Marko Mäkelä
3bc094d32a Remove dead code for pushing down LIMIT to InnoDB FULLTEXT INDEX queries
MySQL 5.7 added code to push down the LIMIT to fulltext search
in InnoDB:

commit 2cd0ebf97e1b265e2282d7ceb5d8dfb663ffc48f
Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date:   Fri May 27 13:49:28 2016 +0530

    Bug    FTS QUERY EXCEEDS RESULT CACHE LIMIT

The code was disabled when MySQL 5.7.9 was merged to MariaDB 10.2.2.

We shall remove the disabled code and unnecessary variables.
2017-10-18 09:07:53 +03:00
Marko Mäkelä
73deafbc17 Remove dict_index_t::is_ngram
When MySQL 5.7 introduced fulltext parser plugins to InnoDB,
it hard-coded the plugin name "ngram" to mean something special.
Because -fsanitize=undefined was issuing warnings for the
assignment in row_merge_create_index() that the value is out of
range for Boolean, we remove this code that was not intended to
be used in MariaDB 10.2.

fts_check_token(): Remove the special logic for N-gram tokens.
2017-05-26 22:45:54 +03:00
Marko Mäkelä
7edadde72e Remove bogus __attribute__((nonnull))
These were reported when running tests after compiling the debug server
with -O3 -fsanitize=undefined in GCC 7.1.0.
2017-05-19 22:58:59 +03:00