Commit graph

3287 commits

Author SHA1 Message Date
Oleksandr Byelkin
b4210f3640 Merge branch '10.0' into 10.1 2018-08-21 10:07:26 +02:00
Oleksandr Byelkin
bcc677bb72 Merge branch '5.5' into 10.0 2018-08-15 16:48:13 +02:00
sachin
68ebfb31f2 MDEV-16166 RBR breaks with HA_ERR_KEY_NOT_FOUND upon DELETE from table...
with spatial index

So the issue is since it is spatial index , at the time of searching index
for key (Rows_log_event::find_row) we use wrong field image we use
Field::itRAW while we should be using Field::itMBR
2018-08-04 01:39:35 +05:30
Sergei Golubchik
bb24663f5a MDEV-13577 slave_parallel_mode=optimistic should not report the mode's specific temporary errors
Revert 7bbe324fc1
Fix the bug differently (in log_event.cc)
Fix the test case to actually fail without the bug fix
2018-06-20 11:10:27 +02:00
Andrei Elkin
7bbe324fc1 MDEV-13577 slave_parallel_mode=optimistic should not report the mode's
specific temporary errors

The optimistic parallel slave's worker thread could face a run-time error due to
the algorithm's specifics which allows for conflicts like the reported
"Can't find record in 'table'".
A typical stack is like

{noformat}
  handler::print_error (this=0x61c00008f8a0, error=149, errflag=0) at handler.cc:3650
  0x0000555555e95361 in write_record (thd=thd@entry=0x62a0000a2208, table=table@entry=0x61f00008ce88, info=info@entry=0x7fffdee356d0) at sql_insert.cc:1944
  0x0000555555ea7767 in mysql_insert (thd=thd@entry=0x62a0000a2208, table_list=0x61b00012ada0, fields=..., values_list=..., update_fields=..., update_values=..., duplic=<optimized out>, ignore=<optimized out>) at sql_insert.cc:1039
  0x0000555555efda90 in mysql_execute_command (thd=thd@entry=0x62a0000a2208) at sql_parse.cc:3927
  0x0000555555f0cc50 in mysql_parse (thd=0x62a0000a2208, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at sql_parse.cc:7449
  0x00005555566d4444 in Query_log_event::do_apply_event (this=0x61200005b9c8, rgi=<optimized out>, query_arg=<optimized out>, q_len_arg=<optimized out>) at log_event.cc:4508
  0x00005555566d639e in Query_log_event::do_apply_event (this=<optimized out>, rgi=<optimized out>) at log_event.cc:4185
  0x0000555555d738cf in Log_event::apply_event (rgi=0x61d0001ea080, this=0x61200005b9c8) at log_event.h:1343
  apply_event_and_update_pos_apply (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080, reason=<optimized out>) at slave.cc:3479
  0x0000555555d8596b in apply_event_and_update_pos_for_parallel (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080) at slave.cc:3623
 0x00005555562aca83 in rpt_handle_event (qev=qev@entry=0x6190000fa088, rpt=rpt@entry=0x62200002bd68) at rpl_parallel.cc:50
 0x00005555562bd04e in handle_rpl_parallel_thread (arg=arg@entry=0x62200002bd68) at rpl_parallel.cc:1258
{noformat}

Here {{handler::print_error}} computes whether to error log the
current error when --log-warnings > 1. The decision flag is consulted
bu {{my_message_sql()}} which can be eventually called.
In the bug case the decision is to log.
However in the optimistic mode slave applier case any conflict is
attempted to resolve with rollback and retry to success. Hence the
logging is at least extraneous.

The case is fixed with adding a new flag {{ME_LOG_AS_WARN}} which
{{handler::print_error}} may propagate further on through {{my_error}}
when the error comes from an optimistically running slave worker thread.

The new flag effectively requests the warning level for the errlog record,
while the thread's DA records the actual error (which is regarded as temporary one
by the parallel slave error handler).
2018-06-12 15:29:16 +03:00
Monty
e744c687ca Merge remote-tracking branch 'origin/10.0' into 10.1 2018-05-24 11:08:02 +03:00
Monty
da71c1bad7 MDEV-16229 Replication aborts with ER_VIEW_SELECT_TMPTABLE after half-failed RENAME
Problem was that detection of temporary tables was all wrong for
RENAME TABLE.
(Temporary tables where opened by top level call to
open_temporary_tables(), which can't detect if a temporary table
was renamed to something and then reused).

Fixed by adding proper parsing of rename list to check against
the current name of a table at each rename stage.
Also change do_rename_temporary() to check against the current
state of temporary tables, not according to the state of start
of RENAME TABLE.
2018-05-22 23:05:48 +03:00
Sergei Golubchik
1bc3899a52 Merge branch '10.0' into 10.1 2018-05-08 17:08:23 +02:00
Sergei Golubchik
34045af03f MDEV-15216 Assertion `! is_set() || m_can_overwrite_status' failed in Diagnostics_area::set_error_status upon operation inside XA
don't implicitly commit or rollback in mysql_admin_table()
unless the statement has CF_IMPLICIT_COMMIT_END flag.
2018-05-08 10:48:13 +02:00
Sergei Golubchik
087ea8f820 de-obfuscate rpl_*_implicit_commit_binlog test 2018-05-08 10:31:35 +02:00
Andrei Elkin
9005108234 MDEV-14721 Big transaction events get lost on semisync master when
replicate_events_marked_for_skip=FILTER_ON_MASTER

[Note this is a cherry-pick from 10.2 branch.]

When events of a big transaction are binlogged offsetting over 2GB from
the beginning of the log the semisync master's dump thread
lost such events.
The events were skipped by the Dump thread that found their skipping
status erroneously.

The current fixes make sure the skipping status is computed correctly.
The test verifies them simulating the 2GB offset.
2018-03-12 17:07:06 +02:00
Marko Mäkelä
016caa3d20 Merge 10.0 into 10.1 2018-01-02 21:57:22 +02:00
Marko Mäkelä
51e4650ed0 Merge 5.5 into 10.0 2018-01-02 21:52:46 +02:00
Sergei Golubchik
1300627a5d MDEV-14309 MTR tests require perl-Env which is not always in the default installation
* don't use Env module in tests, use $ENV{xxx} instead
* collateral changes:
** $file in the error message was unset
** $file in the other error message was unset too :)
** source file arguments are conventionally upper-cased
** abort the test (die) on error, don't just echo/exit
2017-12-27 23:17:28 +01:00
Monty
d7b0b8ddac MDEV-10688 rpl.rpl_row_log_innodb failed in buildbot
Problem was that Binlog_checkpoint can happen at random times.
Fixed by not write binlog_checkpoint for the rpl_log test.

Other things:
- Removed not used variable "$keep_gtid_events"
- Added option for show_binlog_events to skip binlog_checkpoint
2017-12-03 15:21:53 +02:00
Andrei Elkin
aae4932775 MDEV-12012/MDEV-11969 Can't remove GTIDs for a stale GTID Domain ID
As reported in MDEV-11969 "there's no way to ditch knowledge" about some
domain that is no longer updated on a server. Besides being of annoyance to
clutter output in DBA console stale domains can prevent the slave
to connect the master as MDEV-12012 witnesses.
What domain is obsolete must be evaluated by the user (DBA) according
to whether the domain info is still relevant and will the domain ever
receive any update.

This patch introduces a method to discard obsolete gtid domains from
the server binlog state. The removal requires no event group from such
domain present in existing binlog files though. If there are any the
containing logs must be first PURGEd in order for

  FLUSH BINARY LOGS DELETE_DOMAIN_ID=(list-of-domains)

succeed. Otherwise the command returns an error.

The list of obsolete domains can be computed through
intersecting two sets - the earliest (first) binlog's Gtid_list
and the current value of @@global.gtid_binlog_state - and extracting
the domain id components from the intersection list items.
The new DELETE_DOMAIN_ID featured FLUSH continues to rotate binlog
omitting the deleted domains from the active binlog file's Gtid_list.
Notice though when the command is ineffective - that none of requested to delete
domain exists in the binlog state - rotation does not occur.

Obsolete domain deletion is not harmful for connected slaves as long
as master side binlog files *purge* is synchronized with FLUSH-DELETE_DOMAIN_ID.
The slaves must have the last event from purged files processed as usual,
in order not to bump later into requesting a gtid from a file which
was already gone.
While the command is not replicated (as ordinary FLUSH BINLOG LOGS is)
slaves, even though having extra domains, won't suffer from reconnection errors
thanks to master-slave gtid connection protocol allowing the master
to be ignorant about a gtid domain.
Should at failover such slave to be promoted into master role it may run
the ex-master's

 FLUSH BINARY LOGS DELETE_DOMAIN_ID=(list-of-domains)

to clean its own binlog state.

NOTES.
  suite/perfschema/r/start_server_low_digest.result
is re-recorded as consequence of internal parser codes changes.
2017-11-15 22:26:32 +02:00
Sergei Golubchik
f4f48e0621 MDEV-12672 Replicated TIMESTAMP fields given wrong value near DST change
Implement a special Copy_field method for timestamps, that copies
timestamps without converting them to MYSQL_TIME (the conversion
is lossy around DST change dates).
2017-09-21 22:03:21 +02:00
Sergei Golubchik
2e3a16e366 Merge branch '10.0' into 10.1 2017-09-21 22:02:21 +02:00
Sergei Golubchik
b7434bacbd include/master-slave.inc must always be included last 2017-09-20 18:17:50 +02:00
Vicențiu Ciorbaru
ec6042bda0 Merge branch '10.0' into 10.1 2017-09-19 12:06:50 +03:00
Alexander Barkov
434e283507 MDEV-13685 Can not replay binary log due to Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8mb4_general_ci,COERCIBLE) for operation 'concat' 2017-09-15 12:25:06 +04:00
Sergei Golubchik
8e8d42ddf0 Merge branch '10.0' into 10.1 2017-08-08 10:18:43 +02:00
Sergei Golubchik
496cea45e2 update error messages for 10.0 2017-07-27 12:42:40 +02:00
Vicențiu Ciorbaru
786ad0a158 Merge remote-tracking branch 'origin/5.5' into 10.0 2017-07-25 00:41:54 +03:00
Sergei Golubchik
9a5fe1f4ea Merge remote-tracking branch 'mysql/5.5' into 5.5 2017-07-18 14:59:10 +02:00
Sergei Golubchik
9e11e055ce Merge branch '10.0' into 10.1 2017-07-07 11:30:03 +02:00
Kristian Nielsen
228479a28c MDEV-8075: DROP TEMPORARY TABLE not marked as ddl, causing optimistic parallel replication to fail
CREATE/DROP TEMPORARY TABLE are not safe to optimistically replicate in
parallel with other transactions, so they need to be marked as "ddl" in the
binlog.

This was already done for stand-alone CREATE/DROP TEMPORARY. But temporary
tables can also be created and dropped inside a BEGIN...END transaction, and
such transactions were not marked as ddl. Nor was the DROP TEMPORARY TABLE
statement emitted implicitly when a client connection is closed.

So this patch adds such ddl mark for the missing cases.

The difference to Kristian's original patch is mainly a fix in
mysql_trans_commit_alter_copy_data() to remember the unsafe_rollback_flags
over the temporary commit.
2017-07-03 11:16:13 +03:00
Andrei Elkin
946a07e8a8 Fix for MDEV-9670 server_id mysteriously set to 0
Problem was that in a circular replication setup the master remembers
position to events it has generated itself when reading from a slave.
If there are no new events in the queue from the slave, a
Gtid_list_log_event is generated to remember the last skipped event.
The problem happens if there is a network delay and we generate a
Gtid_list_log_event in the middle of the transaction, in which case there
will be an implicit comment and a new transaction with serverid=0 will be
logged.

The fix was to not generate any Gtid_list_log_events in the middle of a
transaction.
2017-07-02 19:47:30 +03:00
Venkatesh Duggirala
bb9e547a86 Bug#18950197 RPL_SEMI_SYNC_UNINSTALL_PLUGIN FAILS BECAUSE
RPL_SEMI_SYNC_MASTER_CLIENTS=1

Analysis: Uninstalling rpl_semi_sync_slave on slave
          will trigger removing the slave logic on Master which
          will reduce Rpl_semi_sync_master_clients by one number.
          But it happens asynchronously on Master. Having assert
          to check this value with zero will have problems on
          slow pb2 machines.

Fix: Change assert into wait_for_status_var condition.
2017-05-25 12:39:20 +05:30
Marko Mäkelä
b61700c221 Merge 10.0 into 10.1 2017-05-23 08:59:03 +03:00
Sergei Golubchik
725e47bfb5 Merge branch '5.5' into 10.0 2017-05-20 00:59:40 +02:00
Sachin Setiya
7d57ba6e28 MDEV-11092 :- Fix Previous commit of MDEV-11092 2017-05-19 13:02:45 +05:30
Sachin Setiya
b5cdf01404 MDEV-11092 Assertion `!writer.checksum_len || writer.remains == 0' failed
Problem:-
This crash happens because logged stmt is quite big and while writing
Annotate_rows_log_event it throws EFBIG error  but we ignore this error
and do not call cache_data->set_incident().

Solution:-
When we normally write Binlog_log_event we check for error EFBIG, but we did
do this for Annotate_rows_log_event. We check for this error and call
cache_data->set_incident() accordingly.

# Conflicts:
#	sql/log.cc
2017-05-18 17:13:37 +05:30
Sergei Golubchik
386ef08704 MDEV-12233 main.mysql_upgrade_noengine fails in buildbot on ppc64le
verify that tests don't leave mysql_upgrade_info in datadir
2017-03-21 11:37:24 +01:00
Marko Mäkelä
ad0c218a44 Merge 10.0 into 10.1
Also, implement MDEV-11027 a little differently from 5.5 and 10.0:

recv_apply_hashed_log_recs(): Change the return type back to void
(DB_SUCCESS was always returned).

Report progress also via systemd using sd_notifyf().
2017-03-09 08:53:08 +02:00
Vicențiu Ciorbaru
1acfa942ed Merge branch '5.5' into 10.0 2017-03-03 01:37:54 +02:00
Sujatha Sivakumar
e619295e1b Bug#24901077: RESET SLAVE ALL DOES NOT ALWAYS RESET SLAVE
Description:
============
If you have a relay log index file that has ended up with
some relay log files that do not exists, then RESET SLAVE
ALL is not enough to get back to a clean state.

Analysis:
=========
In the bug scenario slave server is in stopped state and
some of the relay logs got deleted but the relay log index
file is not updated.

During slave server restart replication initialization fails
as some of the required relay logs are missing. User
executes RESET SLAVE/RESET SLAVE ALL command to start a
clean slave. As per the documentation RESET SLAVE command
clears the master info and relay log info repositories,
deletes all the relay log files, and starts a new relay log
file. But in a scenario where the slave server's
Relay_log_info object is not initialized slave will not
purge the existing relay logs. Hence the index file still
remains in a bad state. Users will not be able to start
the slave unless these files are cleared.

Fix:
===
RESET SLAVE/RESET SLAVE ALL commands should do the cleanup
even in a scenario where Relay_log_info object
initialization failed.

Backported a flag named 'error_on_rli_init_info' which is
required to identify slave's Relay_log_info object
initialization failure. This flag exists in MySQL-5.6
onwards as part of BUG#14021292 fix.

During RESET SLAVE/RESET SLAVE ALL execution this flag
indicates the Relay_log_info initialization failure.
In such a case open the relay log index/relay log files
and do the required clean up.
2017-02-28 10:00:51 +05:30
Marko Mäkelä
3c47ed4849 Merge 10.0 into 10.1 2017-02-20 14:02:40 +02:00
Elena Stepanova
343ba58562 MDEV-10631 rpl.rpl_mdev6386 failed in buildbot
The failure happens due to a race condition between processing
a row event (INSERT) and an automatically generated event
DROP TEMPORARY TABLE. Even though DROP has a higher GTID, it can
become visible in @@gtid_slave_pos before the row event with
a lower GTID has been committed. Since the test makes the slave
to synchronize with the master using GTID, the waiting stops
as soon as GTID of the DROP TEMPORARY TABLE becomes visible,
and if changes from the previous event haven't been applied yet,
the error occurs.

According to Kristian (see the comment to MDEV-10631), the real
problem is that DROP TEMPORARY TABLE is logged in the row mode
at all. For this particular test, since DROP does not do anything,
nothing prevents it from competing with the prior transaction.

The workaround for the test is to add a meaningful event
after DROP TEMPORARY TABLE, so that the slave would wait on its
GTID instead of the one from DROP.

Additionally (unrelated to this problem) removed FLUSH TABLES,
which, as the comment stated, should have been removed after
MDEV-6403 was fixed.
2017-02-20 01:48:26 +02:00
Elena Stepanova
b70cd26d73 MDEV-11668 rpl.rpl_heartbeat_basic fails sporadically in buildbot
On a slow builder, a delay between binlog events on master could
occur, which would cause a heartbeat which is not expected by the
test. The solution is to monitor the timing of binlog events
on the master and only perform the heartbeat check if no critical
delays have happened.

Additionally, an unused variable was removed (this change is
unrelated to the bugfix).
2017-02-17 00:57:24 +02:00
Sachin Setiya
86ca1357b0 Revert "MDEV-7409 On RBR, extend the PROCESSLIST info to include at least the name of the recently used table"
This reverts commit 15f46d5174.
2017-01-25 17:15:14 +05:30
Sachin Setiya
15f46d5174 MDEV-7409 On RBR, extend the PROCESSLIST info to include at least the name of the recently used table
When RBR is used, add the db name to db Field and table name to Status
Field  of the "SHOW FULL PROCESSLIST" command for SQL thread.
2017-01-23 23:03:19 +05:30
Alexey Botchkov
30a9ac4250 MDEV-10956 Strict Password Validation Breaks Replication.
strict_password_validation variable now has
        no effect in the slave thread.
2017-01-17 15:32:41 +04:00
Vicențiu Ciorbaru
1e192e901c Post merge review fixes
* Remove duplicate lines from tests
* Use thd instead of current_thd
* Remove extra wsrep_binlog_format_names
* Correctly merge union patch from 5.5 wrt duplicate rows.
* Correctly merge SELinux changes into 10.1
2017-01-17 04:16:38 +02:00
Vicențiu Ciorbaru
7b44c31be8 Merge remote-tracking branch 'origin/10.1' into 10.1
Extra merge due to more extra commits during merge process
2017-01-16 12:18:21 +02:00
Vicențiu Ciorbaru
8e15768731 Merge branch '10.0' into 10.1 2017-01-16 03:18:14 +02:00
Marko Mäkelä
a9d00db155 MDEV-11799 InnoDB can abort if the doublewrite buffer
contains a bad and a good copy

Clean up the InnoDB doublewrite buffer code.

buf_dblwr_init_or_load_pages(): Do not add empty pages to the buffer.

buf_dblwr_process(): Do consider changes to pages that are all zero.
Do not abort when finding a corrupted copy of a page in the doublewrite
buffer, because there could be multiple copies in the doublewrite buffer,
and only one of them needs to be good.
2017-01-15 18:56:56 +02:00
vicentiu
e9aed131ea Merge remote-tracking branch 'origin/5.5' into 10.0 2017-01-06 17:09:59 +02:00
Marko Mäkelä
8049d2e9d9 Merge 10.0 into 10.1 2017-01-05 20:32:15 +02:00
Elena Stepanova
9bf92706d1 MDEV-8518 rpl.sec_behind_master-5114 fails sporadically in buildbot
- fix the test to avoid false-negatives before MDEV-5114 patch;
- fix the race condition which made the test fail on slow builders
2017-01-04 14:50:10 +02:00