Commit graph

186135 commits

Author SHA1 Message Date
Marko Mäkelä
f11d425a15 MDEV-20591: Follow-up fix
calc_field_event_length(): For type=MYSQL_TYPE_BLOB and meta==0,
return 0 instead of *ptr+1. This was noted by -Wimplicit-fallthrough.
2019-10-09 13:08:16 +03:00
Oleksandr Byelkin
896b869685 MDEV-19238 Mariadb spider - crashes on where null
(fix and explanation came with MDEV-20753 (duplicate of this bug))
2019-10-09 08:55:00 +02:00
Oleksandr Byelkin
b7408be0c3 MDEV-20753: Sequence with limit 0 crashes server
Do not try to push down conditions to engine if query was resolved without tables (and so the engine).
2019-10-09 08:55:00 +02:00
Marko Mäkelä
24232ec12c Merge 10.1 into 10.2 2019-10-09 08:30:23 +03:00
Eugene Kosov
ed0793e096 MDEV-19783: Add more REC_INFO_MIN_REC_FLAG checks
btr_cur_pessimistic_delete(): code changed in a way that allows
to put more REC_INFO_MIN_REC_FLAG assertions inside btr_set_min_rec_mark().
Without that change tests innodb.innodb-table-online,
innodb.temp_table_savepoint and innodb_zip.prefix_index_liftedlimit fail.

Removed basically duplicated page_zip_validate() calls
which fails because of temporary(!) invariant violation.
That fixed innodb_zip.wl5522_debug_zip and
innodb_zip.prefix_index_liftedlimit
2019-10-09 08:29:26 +03:00
Eugene Kosov
99dc40d6ac MDEV-19783 Random crashes and corrupt data in INSTANT-added columns
The bug affects MariaDB Server 10.3 or later, but it makes sense
to improve CHECK TABLE in earlier versions already.

page_validate(): Check REC_INFO_MIN_REC_FLAG in the records.
This allows CHECK TABLE to catch more bugs.
2019-10-09 08:29:26 +03:00
Marko Mäkelä
d480d28f4f Add page_has_prev(), page_has_next(), page_has_siblings()
Until now, InnoDB inefficiently compared the aligned fields
FIL_PAGE_PREV, FIL_PAGE_NEXT to the byte-order-agnostic value FIL_NULL.

This is a backport of 32170f8c6d
from MariaDB Server 10.3.
2019-10-09 08:29:26 +03:00
Seth Shelnutt
b1ac174279 Cast string literal to char* MDEV-20767 2019-10-09 07:20:34 +03:00
Marko Mäkelä
d95f96ad1b Merge 5.5 into 10.1 2019-10-08 12:43:37 +03:00
Sachin Setiya
27664ef29d MDEV-20574 Position of events reported by mysqlbinlog is wrong with encrypted binlogs, SHOW BINLOG EVENTS reports the correct one.
Analysis

Mysqlbinlog output for encrypted binary log
#Q> insert into tab1 values (3,'row 003')
 17:36:35 server id 10221  end_log_pos 980 CRC32 0x53bcb3d3  Table_map: `test`.`tab1` mapped to number 19
# at 940
 17:36:35 server id 10221  end_log_pos 1026 CRC32 0xf2ae5136     Write_rows: table id 19 flags: STMT_END_F

Here we can see Table_map_log_event ends at 980 but Next event starts at 940.
And the reason for that is we do not send START_ENCRYPTION_EVENT to the slave

Solution:-
Send Start_encryption_log_event as Ignorable_log_event to slave(mysqlbinlog),
So that mysqlbinlog can update its log_pos.
Since Slave can request multiple FORMAT_DESCRIPTION_EVENT while master does not
have so We only update slave master pos when master actually have the
FORMAT_DESCRIPTION_EVENT. Similar logic should be applied for START_ENCRYPTION_EVENT.

Also added the test case when new server reads the data from old server which
does not send START_ENCRYPTION_EVENT to slave.

Master Slave Upgrade Scenario.
When Slave is updated first, Slave will have extra logic of handling
START_ENCRYPTION_EVENT But master willnot be sending START_ENCRYPTION_EVENT.
So there will be no issue.
When Master is updated first, It will send  START_ENCRYPTION_EVENT to
slave , But slave will ignore this event in queue_event.
2019-10-08 14:35:34 +05:30
sachin
1e0f09cacb MDEV-16239 Many test in rpl suite fails
Fix rpl_skip_error test.
  We cant reset Slave_skipped_errors(even with FLUSH STATUS), So instead
of absolute slave_skipped_errors we look for delta of slave_skipped_errors
Fix rpl.rpl_binlog_errors and binlog_encryption.rpl_binlog_errors
  We create the $load_file and $load_file2 but we never remove them.
Fix rpl_000011.test
  Instead of real value use delta value , Since flush status wont flush
LONGLONG variable.
Fix rpl_row_find_row_debug
  Instead of searching whole log_error_ file we will use search_pattern_in_file
which runs pattern search only on latest test run , instead of full file.
Fix rpl_ip_mix rpl_ip_mix2
  We should call reset slave all because we also want to reset master_host
otherwise show slave status wont be empty and making repeat N a failure.
Fix rpl_rotate_logs
  First we have to remove master.info file (cleanup) and second we have to
call reset slave all because if we do not call reset slave all then we wont
read master.info file beacuse we already have master config in memory.
And this makes start slave to pass , which shoud fail becuase its permision
is 000
Fix circular_serverid0 test
  The reason is that ++dbug_rows_event_count == 2 in queue_event does
not take --repeat into account. So I have reseted the dbug_rows_event_count
in if body.
2019-10-08 13:34:25 +05:30
Sachin
01bf9f8c3d MDEV-20591 Wrong Number of rows in mysqlbinlog output
calc_field_event_length should accurately calculate the size of BLOB type
fields, Instead of returning just the bytes taken by length it should return
length bytes + actual length.
2019-10-08 13:34:11 +05:30
Marko Mäkelä
db9a4d928d Remove orphan declaration buf_flush_wait_batch_end_wait_only()
The function was declared but not defined in
commit 9d6d1902e0
2019-10-07 17:18:10 +03:00
Sergey Vojtovich
adefaeffcc MDEV-19536 - Server crash or ASAN heap-use-after-free in is_temporary_table /
read_statistics_for_tables_if_needed

Regression after 279a907, read_statistics_for_tables_if_needed() was
called after open_normal_and_derived_tables() failure.

Fixed by moving read_statistics_for_tables() call to a branch of
get_schema_stat_record() where result of open_normal_and_derived_tables()
is checked.

Removed THD::force_read_stats, added read_statistics_for_tables() instead.
Simplified away statistics_for_command_is_needed().
2019-10-07 13:30:22 +04:00
Oleksandr Byelkin
13535b2713 Fix problem with warnings of new compilers. 2019-10-04 11:03:54 +02:00
Eugene Kosov
5e65c67cfc fix clang warning 2019-10-03 17:45:36 +03:00
Sergey Vojtovich
5b2fa078e8 Cleanup mman.h includes
As it is included from my_global.h already.
2019-10-02 20:21:30 +04:00
Daniel Black
716c748f97 MDEV-20684: innodb/query cache use madvise CORE/NOCORE on FreeBSD
This applies to large allocations.

This maps to the way Linux does it in MDEV-10814 except FreeBSD uses
different constants.

Adjust error string to match to implementation.

Tested on FreeBSD-12.0
2019-10-02 20:00:05 +04:00
Sergey Vojtovich
e43791d4dc Cleanup EITS
Moved EITS allocation inside read_statistics_for_tables_if_needed().
Removed redundant is_safe argument.
2019-10-02 15:23:59 +04:00
Vlad Lesin
edda2fd149 MDEV-20703: mariabackup creates binlog files in server binlog directory on --prepare --export step
When "--export" mariabackup option is used, mariabackup starts the server in
bootstrap mode to generate *.cfg files for the certain innodb tables.
The started instance of the server reads options from the file, pointed
out in "--defaults-file" mariabackup option.

If the server uses the same config file as mariabackup, and binlog is
switched on in that config file, then "mariabackup --prepare --export"
will create binary log files in the server's binary log directory, what
can cause issues.

The fix is to add "--skip-log-bin" in mysld options when the server is
started to generate *.cfg files.
2019-10-01 13:57:24 +03:00
Alexander Barkov
7e44c455f4 Merge remote-tracking branch 'origin/10.2' into 10.3 2019-10-01 09:37:40 +04:00
Alexander Barkov
f203245e9e Merge remote-tracking branch 'origin/10.1' into 10.2 2019-10-01 07:11:54 +04:00
Alexander Barkov
4bcf524482 Merge remote-tracking branch 'origin/5.5' into 10.1 2019-10-01 06:07:56 +04:00
Alexander Barkov
d481f69a7d MDEV-20704 An index on a double column erroneously uses prefix compression 2019-10-01 05:55:14 +04:00
Robert Bindar
576a5f091e MDEV-20647 Fix and enable SphinxSE tests 2019-09-30 15:47:09 +03:00
Alexey Botchkov
6c2724fc05 MDEV-19628 JSON with starting double quotes key is not valid.
Make the skip_key a bit faster.
2019-09-30 15:27:48 +04:00
Alexey Botchkov
6ac2a35553 MDEV-19628 JSON with starting double quotes key is not valid.
First character of the key name is just skipped, so the escapement
wasn't handled properly.
2019-09-30 14:43:32 +04:00
Marko Mäkelä
46b785262b Fix -Wunused for CMAKE_BUILD_TYPE=RelWithDebInfo
For release builds, do not declare unused variables.

unpack_row(): Omit a debug-only variable from WSREP diagnostic message.

create_wsrep_THD(): Fix -Wmaybe-uninitialized for the PSI_thread_key.
2019-09-30 12:49:53 +03:00
Sujatha
9b80f9300d MDEV-20645: Replication consistency is broken as workers miss the error notification from an earlier failed group.
Analysis:
========
In general if there are three groups.
1 - Inserts 32 which fails due to local entry '32' on slave.
2 - Inserts 33
3 - Inserts 34

Each group considers itself as a waiter and it waits for prior group 'waitee'.
This is done in 'register_wait_for_prior_event_group_commit'. If there is no
other parallel group being scheduled then no waitee will be there.

Let us assume 3 groups are being scheduled in parallel.

3-> waits for 2-> waits for->1

'1' upon completion it checks is there any registered subsequent waiter. If
so it wakes up the subsequent waiter with its execution status. This execution
status is stored in wakeup_error.

If '1' failed then it sends corresponding wakeup_error to 2. Then '2' aborts
and it propagates error to '3'.  So all further commits are aborted.  This
mechanism works only when all transactions reach a stage where they are
waiting for their prior commit to complete.

In case of optimistic following scenario occurs.

1,2,3 are scheduled in parallel.

3 - Reaches group_commit_code waits for 2 to complete.
1 - errors out sets stop_on_error_sub_id=1.

When a group execution results in error its corresponding sub_id is set to
'stop_on_error_sub_id'. Any new groups queued for execution will check if
their sub_id is > stop_on_error_sub_id.  If it is true their execution will be
skipped as prior group execution failed.  'skip_event_group=1' will be set.
Since the execution of SQL thread is about to stop we just skip execution of
all the following event groups.  We still do all the normal waiting and wakeup
processing between the event groups as a simple way to ensure that everything
is stopped and cleaned up correctly.

Upon error '1' transaction checks for registered waiters. Since no one is
there it simply goes away.

2 - Starts the execution. It checks do I have a waitee.

Since wait_commit_sub_id == entry->last_committed_sub_id no waitee is set.

Secondly: 'entry->stop_on_error_sub_id' is set by '1'st execution.  Now
'handle_parallel_thread' code checks if the current group 'sub_id' is greater
than the 'sub_id' set within 'stop_on_error_sub_id'.

Since the above is true 'skip_event_group=true' is set.  Simply call
'wait_for_prior_commit' to wakeup all waiters.  Group '2' didn't had any
waitee and its execution is skipped.  Hence its wakeup_error=0.It sends a
positive wakeup signal to '3'. Which commits. This results in a missed
transaction. i.e 33 is missed and 34 is committed.

Fix:
===
When a worker learns that an earlier transaction execution has failed, and it
should not proceed for further execution, it should mark its own execution
status as failed so that it alerts its followers to abort as well.
2019-09-30 13:22:37 +05:30
Julius Goryavsky
bc70862e13 MDEV-20614: Syntax error, and option put in wrong place
A syntax error in the mysqld_multi.sh script has been fixed
here + a "--defaults-group-suffix" option has been moved to
the top of the mysqld options list.
2019-09-27 17:14:10 +02:00
Marko Mäkelä
2911a9a693 Merge 10.2 into 10.3 2019-09-27 15:56:15 +03:00
Thirunarayanan Balathandayuthapani
c76873f23d MDEV-20688 Recovery crashes after unnecessarily reading a corrupted page
The test encryption.innodb-redo-badkey was accidentally disabled
until commit 23657a2101 enabled
it recently. Once it was enabled, it started failing randomly.

recv_recover_corrupt_page(): Do not assume that any redo log exists
for the page. A page may be unnecessarily read by read-ahead.
When noting the corruption, reset recv_addr->state to RECV_PROCESSED,
so that even if the same page is re-read again, we will only
decrement recv_sys->n_addrs once.
2019-09-27 17:46:10 +05:30
Marko Mäkelä
d874cdeccc dict_load_table(): Remove constant parameter cached=true
Spotted by Thirunarayanan Balathandayuthapani.
2019-09-27 14:29:22 +03:00
Marko Mäkelä
718fcee0a3 Reduce rw_lock_debug_mutex contention
rw_lock_own(), rw_lock_own_flagged(): Traverse the rw_lock_t::debug_list
only after quickly checking if the thread is holding X-latch or SX-latch.
2019-09-27 14:22:59 +03:00
Sergei Golubchik
677cc64428 chkconfig in RPM server scriptlets
chkconfig --add and --del [might] invoke /sbin/insserv
and even if chkconfig exists, insserv might not (SLES15).

Ignore chkconfig --del errors - it's a "best effort" cleanup anyway
2019-09-27 13:20:11 +02:00
Marko Mäkelä
4ec0c346b8 Remove a useless large test, and add a debug assertion
The test innodb_fts.fulltext_table_evict was only creating 1000 tables
with fulltext indexes, only to check that no tables with fulltext
indexes are being evicted.

The reason why tables containing fulltext indexes cannot be evicted is
that fts_optimize_init() invokes dict_table_prevent_eviction().
2019-09-27 14:05:39 +03:00
Marko Mäkelä
ca9e0089d5 MDEV-19740: Fix GCC 9.2.1 -Wmaybe-uninitialized on AMD64
For CMAKE_BUILD_TYPE=Debug, the default MYSQL_MAINTAINER_MODE=AUTO
implies -Werror along with other flags in cmake/maintainer.cmake,
which would break the debug builds when CMAKE_CXX_FLAGS include -O2.

This fix includes a backport of 6dd3f24090
from MariaDB 10.3.
2019-09-27 10:43:23 +03:00
Marko Mäkelä
46facaedbf Fix GCC 9 -Wmaybe-uninitialized
Always initialize ScopedStatementReplication::saved_binlog_format,
so that GCC cannot emit a bogus warning about
ScopedStatementReplication::~ScopedStatementReplication() using the
variable.

The code was originally introduced in
commit d998da0306.
2019-09-26 15:43:51 +03:00
Marko Mäkelä
1cf0694d10 Merge 10.2 into 10.3 2019-09-26 13:18:57 +03:00
Marko Mäkelä
3e4931cdf3 MDEV-20675 Crash in SHOW ENGINE INNODB STATUS with innodb_force_recovery=5
lock_print_info::operator(): Do not dereference purge_sys.query in case
it is NULL. We would not initialize purge_sys if innodb_force_recovery
is set to 5 or 6.

The test case will be added by merge from 10.2.
2019-09-26 13:18:22 +03:00
Marko Mäkelä
2d6719d7ee MDEV-19514 preparation: Extend innodb.innodb-change-buffer-recovery
Test innodb_read_only startup (which will be refused after a crash),
and test also innodb_force_recovery=5, and extract some change buffer
merge statistics. Omit any statistics about delete (purge) buffering,
because purge could happen at any time.

Use the sequence storage engine for populating the table.
2019-09-26 13:03:40 +03:00
Marko Mäkelä
23d675453f MDEV-20675 Crash in SHOW ENGINE INNODB STATUS with innodb_force_recovery=5
Add a test case. MariaDB Server 10.2 is not affected.
2019-09-26 13:03:40 +03:00
Marko Mäkelä
574ff87f2a Merge 10.1 into 10.2 2019-09-26 12:36:01 +03:00
Julius Goryavsky
144217339d MDEV-20614: Syntax error, and option put in wrong place
A syntax error in the mysqld_multi.sh script has been fixed
here + a "--defaults-group-suffix" option has been moved to
the top of the mysqld options list.
2019-09-26 11:11:43 +02:00
Marko Mäkelä
516f7c111b Speed up main.sum_distinct-big
Eliminate one InnoDB table with 128*16384 rows, and use
the sequence engine instead. Also, run everything in a single
transaction, to prevent purge from running concurrently
unnecessarily. (Starting with MariaDB Server 10.3, purge would
reset the DB_TRX_ID after INSERT.)
2019-09-25 11:29:21 +03:00
Alexander Barkov
b6bb64e54a Merge remote-tracking branch 'origin/10.2' into 10.3 2019-09-24 23:05:09 +04:00
Vicențiu Ciorbaru
0e50ee6f28 Remove --basedir arg from systemd startup scripts
The arg was introduced as part of 75bcf1f9ad
to fix a SELinux problem caused by mysqld_safe accessing files it should
not be via the my_which function.

The root cause for this was fixed in 10.3, via
355ee6877b which eliminated the my_which
function from mysqld_safe entirely. Thus, in 10.3, this --basedir flag
is not necessary.
2019-09-24 20:43:32 +03:00
Alexander Barkov
102bc7beb0 Fixing tests according to MDEV-20655 maturity stable for user_variables 2019-09-24 19:24:32 +04:00
Alexander Barkov
0a359d7627 Merge remote-tracking branch 'origin/10.1' into 10.2 2019-09-24 19:19:25 +04:00
Rasmus Johansson
fa9e012a34
MDEV-20655 maturity stable for user_variables 2019-09-24 14:37:01 +03:00