mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2026-05-14 19:07:15 +02:00
Code Issues Projects Releases Packages Wiki Activity

BUG#12400313 RELAY_LOG_SPACE_LIMIT IS NOT WORKING IN MANY CASES

Browse source 
BUG#64503: mysql frequently ignores --relay-log-space-limit

When the SQL thread goes to sleep, waiting for more events, it sets
the flag ignore_log_space_limit to true. This gives the IO thread a
chance to queue some more events and ultimately the SQL thread will be
able to purge the log once it is rotated. By then the SQL thread
resets the ignore_log_space_limit to false. However, between the time
the SQL thread has set the ignore flag and the time it resets it, the
IO thread will be queuing events in the relay log, possibly going way
over the limit.

This patch makes the IO and SQL thread to synchronize when they reach
the space limit and only ask for one event at a time. Thus the SQL
thread sets ignore_log_space_limit flag and the IO thread resets it to
false everytime it processes one more event. In addition, everytime
the SQL thread processes the next event, and the limit has been
reached, it checks if the IO thread should rotate. If it should, it
instructs the IO thread to rotate, giving the SQL thread a chance to
purge the logs (freeing space). Finally, this patch removes the
resetting of the ignore_log_space_limit flag from purge_first_log,
because this is now reset by the IO thread every time it processes the
next event when the limit has been reached.

If the SQL thread is in a transaction, it cannot purge so, there is no
point in asking the IO thread to rotate. The only thing it can do is
to ask for more events until the transaction is over (then it can ask
the IO to rotate and purge the log right away). Otherwise, there would
be a deadlock (SQL would not be able to purge and IO thread would not
be able to queue events so that the SQL would finish the transaction).
This commit is contained in:
Luis Soares 2012-03-12 12:28:27 +00:00
commit 4a6c4d8603
6 changed files with 191 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

92
sql/slave.cc
View file

@ -1450,6 +1450,54 @@ Waiting for the slave SQL thread to free enough relay log space");
!(slave_killed=io_slave_killed(thd,mi)) &&
!rli->ignore_log_space_limit)
pthread_cond_wait(&rli->log_space_cond, &rli->log_space_lock);
/*
Makes the IO thread read only one event at a time
until the SQL thread is able to purge the relay
logs, freeing some space.
Therefore, once the SQL thread processes this next
event, it goes to sleep (no more events in the queue),
sets ignore_log_space_limit=true and wakes the IO thread.
However, this event may have been enough already for
the SQL thread to purge some log files, freeing
rli->log_space_total .
This guarantees that the SQL and IO thread move
forward only one event at a time (to avoid deadlocks),
when the relay space limit is reached. It also
guarantees that when the SQL thread is prepared to
rotate (to be able to purge some logs), the IO thread
will know about it and will rotate.
NOTE: The ignore_log_space_limit is only set when the SQL
thread sleeps waiting for events.
*/
if (rli->ignore_log_space_limit)
{
#ifndef DBUG_OFF
{
char llbuf1[22], llbuf2[22];
DBUG_PRINT("info", ("log_space_limit=%s "
"log_space_total=%s "
"ignore_log_space_limit=%d "
"sql_force_rotate_relay=%d",
llstr(rli->log_space_limit,llbuf1),
llstr(rli->log_space_total,llbuf2),
(int) rli->ignore_log_space_limit,
(int) rli->sql_force_rotate_relay));
}
#endif
if (rli->sql_force_rotate_relay)
{
rotate_relay_log(rli->mi);
rli->sql_force_rotate_relay= false;
}
rli->ignore_log_space_limit= false;
}
thd->exit_cond(save_proc_info);
DBUG_RETURN(slave_killed);
}
@ -4260,19 +4308,45 @@ static Log_event* next_event(Relay_log_info* rli)
constraint, because we do not want the I/O thread to block because of
space (it's ok if it blocks for any other reason (e.g. because the
master does not send anything). Then the I/O thread stops waiting
and reads more events.
The SQL thread decides when the I/O thread should take log_space_limit
into account again : ignore_log_space_limit is reset to 0
in purge_first_log (when the SQL thread purges the just-read relay
log), and also when the SQL thread starts. We should also reset
ignore_log_space_limit to 0 when the user does RESET SLAVE, but in
fact, no need as RESET SLAVE requires that the slave
and reads one more event and starts honoring log_space_limit again.
If the SQL thread needs more events to be able to rotate the log (it
might need to finish the current group first), then it can ask for one
more at a time. Thus we don't outgrow the relay log indefinitely,
but rather in a controlled manner, until the next rotate.
When the SQL thread starts it sets ignore_log_space_limit to false.
We should also reset ignore_log_space_limit to 0 when the user does
RESET SLAVE, but in fact, no need as RESET SLAVE requires that the slave
be stopped, and the SQL thread sets ignore_log_space_limit to 0 when
it stops.
*/
pthread_mutex_lock(&rli->log_space_lock);
// prevent the I/O thread from blocking next times
rli->ignore_log_space_limit= 1;
/*
If we have reached the limit of the relay space and we
are going to sleep, waiting for more events:
1. If outside a group, SQL thread asks the IO thread
to force a rotation so that the SQL thread purges
logs next time it processes an event (thus space is
freed).
2. If in a group, SQL thread asks the IO thread to
ignore the limit and queues yet one more event
so that the SQL thread finishes the group and
is are able to rotate and purge sometime soon.
*/
if (rli->log_space_limit &&
rli->log_space_limit < rli->log_space_total)
{
/* force rotation if not in an unfinished group */
rli->sql_force_rotate_relay= !rli->is_in_group();
/* ask for one more event */
rli->ignore_log_space_limit= true;
}
/*
If the I/O thread is blocked, unblock it. Ok to broadcast
after unlock, because the mutex is only destroyed in