mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-16 12:02:42 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/sql/rpl_mi.cc
Michael Widenius 32f81bab7d WL#3262 add mutex lock order checking to safemutex (also called safe_mutex_deadlock_detector) 
This writes a warning on stderr if one uses mutex in different order,
like if one in one case would lock mutex in the order A,B and in another case
would lock mutex in the order B,A
This is inspired by and loosely based on the LOCKDEP patch by Jonas
Wrong mutex order is either fixed or mutex are marked with MYF_NO_DEADLOCK_DETECTION
if used inconsistently (need to be fixed by server team)

KNOWN_BUGS.txt:
  Added information that one need to dump and restore Maria tables
include/hash.h:
  Added prototype function for walking over all elements in a hash
include/my_pthread.h:
  Added my_pthread_mutex_init() and my_pthread_mutex_lock(); These should be used if one wants to disable mutex order checking.
  Changed names of the nonposix mutex_init functions to not conflict with my_phread_mutex_init()
  Added and extended structures for mutex deadlock detection.
  New arguments to sage_mutex_init() and safe_mutex_lock() to allow one to disable mutex order checking.
  Added variable 'safe_mutex_deadlock_detector' to enable/disable deadlock detection for all pthread_mutex_init()
mysys/Makefile.am:
  Added cleaning of test files
  Added test_thr_mutex
mysys/hash.c:
  Added hash_iterate() to iterate over all elements in a hash
  More comments
mysys/my_init.c:
  Added calls to destory all mutex uses by mysys()
  Added waiting for threads to end before calling TERMINATE() to list not freed memory
mysys/my_pthread.c:
  Changed names to free my_pthread_mutex_init() for mutex-lock-order-checking
mysys/my_sleep.c:
  Fixed too long wait if using 1000000L as argument
mysys/my_thr_init.c:
  Mark THR_LOCK_threads and THR_LOCK_malloc to not have mutex deadlock detection.
  (We can't have it enabled for this as these are internal mutex used by the detector  
  Call my_thread_init() early as we need thread specific variables enabled for the following pthread_mutex_init()
  Move code to wait for threads to end to my_wait_for_other_threads_to_die()
  Don't destroy mutex and conditions unless all threads have died
  Added my_thread_destroy_mutex() to destroy all mutex used by the mysys thread system
  Name the thread specific mutex as "mysys_var->mutex"
  Added my_thread_var_mutex_in_use() to return pointer to mutex in use or 0 if thread variables are not initialized
mysys/mysys_priv.h:
  Added prototypes for functions used internally with mutex-wrong-usage detection
mysys/thr_mutex.c:
  Added runtime detection of mutex used in conflicting order
  See WL#3262 or test_thr_mutex.c for examples
  The base idea is for each mutex have two hashes:
  - mutex->locked_mutex points to all mutex used after this one
  - mutex->used_mutex points to all mutex which has this mutex in it's mutex->locked_mutex
  There is a wrong mutex order if any mutex currently locked before this mutex is in the mutex->locked_mutex hash
sql/event_queue.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/event_scheduler.cc:
  Declare the right order to take the mutex
sql/events.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/ha_ndbcluster_binlog.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/log.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/mysqld.cc:
  Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
  This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
  Added call to free sp cache.
  Add destruction of LOCK_server_started and COND_server_started.
  Added register_mutex_order() function to register in which order mutex should be taken
  (to initiailize mutex_deadlock_detector).
  Added option to turn off safe_mutex_deadlock_detector
sql/protocol.cc:
  Fixed wrong argument to DBUG_PRINT (found by valgrind)
sql/rpl_mi.cc:
  Mark mutex used inconsistently (need to be fixed by server team)
sql/set_var.cc:
  Remove wrong locking of LOCK_global_system_variables when reading and setting log variables
  (would cause inconsistent mutex order).
  Update global variables outside of logger.unlock() as LOCK_global_system_variables has to be taken before logger locks
  Reviewed by gluh
sql/sp_cache.cc:
  Added function to destroy mutex used by sp cache
sql/sp_cache.h:
  Added function to destroy mutex used by sp cache
sql/sql_class.cc:
  Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
  This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
  Register order in which LOCK_delete and mysys_var->mutex is taken
sql/sql_insert.cc:
  Give a name for Delayed_insert::mutex
  Mark mutex used inconsistently (need to be fixed by server team)
  Move closing of tables outside of di->mutex (to avoid wrong mutex order)
sql/sql_show.cc:
  Don't keep LOCK_global_system_variables locked over value->show_type() as this leads to wrong mutex order
storage/innobase/handler/ha_innodb.cc:
  Disable safe_muted_deadlock_detector for innobase intern mutex (to speed up page cache initialization)
storage/maria/ha_maria.cc:
  Added flag to ha_maria::info() to signal if we need to lock table share or not.
  This is needed to avoid locking mutex in wrong order
storage/maria/ha_maria.h:
  Added flag to ha_maria::info() to signal if we need to lock table share or not.
storage/maria/ma_close.c:
  Destroy key_del_lock
  Simplify freeing ftparser_param
storage/maria/ma_key.c:
  Better comment
storage/maria/ma_loghandler.c:
  Mark mutex used inconsistently (need to be fixed by sanja)
storage/maria/ma_state.c:
  More comments
storage/maria/ma_test1.c:
  Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
storage/maria/ma_test2.c:
  Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
2008-12-03 00:02:52 +02:00

429 lines
14 KiB
C++
Raw Blame History

/* Copyright (C) 2000-2003 MySQL AB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
#include <my_global.h> // For HAVE_REPLICATION
#include "mysql_priv.h"
#include <my_dir.h>
#include "rpl_mi.h"
#ifdef HAVE_REPLICATION
// Defined in slave.cc
int init_intvar_from_file(int* var, IO_CACHE* f, int default_val);
int init_strvar_from_file(char *var, int max_size, IO_CACHE *f,
const char *default_val);
Master_info::Master_info()
:Slave_reporting_capability("I/O"),
ssl(0), ssl_verify_server_cert(0), fd(-1), io_thd(0), inited(0),
abort_slave(0),slave_running(0),
slave_run_id(0)
{
host[0] = 0; user[0] = 0; password[0] = 0;
ssl_ca[0]= 0; ssl_capath[0]= 0; ssl_cert[0]= 0;
ssl_cipher[0]= 0; ssl_key[0]= 0;
bzero((char*) &file, sizeof(file));
/*
We have to use MYF_NO_DEADLOCK_DETECTION because mysqld doesn't
lock run_lock and data_lock consistently.
Should be fixed as this can easily lead to deadlocks
*/
my_pthread_mutex_init(&run_lock, MY_MUTEX_INIT_FAST,
"Master_info::run_lock", MYF_NO_DEADLOCK_DETECTION);
my_pthread_mutex_init(&data_lock, MY_MUTEX_INIT_FAST,
"Master_info::data_lock", MYF_NO_DEADLOCK_DETECTION);
pthread_cond_init(&data_cond, NULL);
pthread_cond_init(&start_cond, NULL);
pthread_cond_init(&stop_cond, NULL);
#ifdef SAFE_MUTEX
/* Define mutex order for locks to find wrong lock usage */
pthread_mutex_lock(&data_lock);
pthread_mutex_lock(&run_lock);
pthread_mutex_unlock(&run_lock);
pthread_mutex_unlock(&data_lock);
#endif
}
Master_info::~Master_info()
{
pthread_mutex_destroy(&run_lock);
pthread_mutex_destroy(&data_lock);
pthread_cond_destroy(&data_cond);
pthread_cond_destroy(&start_cond);
pthread_cond_destroy(&stop_cond);
}
void init_master_info_with_options(Master_info* mi)
{
DBUG_ENTER("init_master_info_with_options");
mi->master_log_name[0] = 0;
mi->master_log_pos = BIN_LOG_HEADER_SIZE; // skip magic number
if (master_host)
strmake(mi->host, master_host, sizeof(mi->host) - 1);
if (master_user)
strmake(mi->user, master_user, sizeof(mi->user) - 1);
if (master_password)
strmake(mi->password, master_password, MAX_PASSWORD_LENGTH);
mi->port = master_port;
mi->connect_retry = master_connect_retry;
mi->ssl= master_ssl;
if (master_ssl_ca)
strmake(mi->ssl_ca, master_ssl_ca, sizeof(mi->ssl_ca)-1);
if (master_ssl_capath)
strmake(mi->ssl_capath, master_ssl_capath, sizeof(mi->ssl_capath)-1);
if (master_ssl_cert)
strmake(mi->ssl_cert, master_ssl_cert, sizeof(mi->ssl_cert)-1);
if (master_ssl_cipher)
strmake(mi->ssl_cipher, master_ssl_cipher, sizeof(mi->ssl_cipher)-1);
if (master_ssl_key)
strmake(mi->ssl_key, master_ssl_key, sizeof(mi->ssl_key)-1);
/* Intentionally init ssl_verify_server_cert to 0, no option available */
mi->ssl_verify_server_cert= 0;
DBUG_VOID_RETURN;
}
enum {
LINES_IN_MASTER_INFO_WITH_SSL= 14,
/* 5.1.16 added value of master_ssl_verify_server_cert */
LINE_FOR_MASTER_SSL_VERIFY_SERVER_CERT= 15,
/* Number of lines currently used when saving master info file */
LINES_IN_MASTER_INFO= LINE_FOR_MASTER_SSL_VERIFY_SERVER_CERT
};
int init_master_info(Master_info* mi, const char* master_info_fname,
const char* slave_info_fname,
bool abort_if_no_master_info_file,
int thread_mask)
{
int fd,error;
char fname[FN_REFLEN+128];
DBUG_ENTER("init_master_info");
if (mi->inited)
{
/*
We have to reset read position of relay-log-bin as we may have
already been reading from 'hotlog' when the slave was stopped
last time. If this case pos_in_file would be set and we would
get a crash when trying to read the signature for the binary
relay log.
We only rewind the read position if we are starting the SQL
thread. The handle_slave_sql thread assumes that the read
position is at the beginning of the file, and will read the
"signature" and then fast-forward to the last position read.
*/
if (thread_mask & SLAVE_SQL)
{
my_b_seek(mi->rli.cur_log, (my_off_t) 0);
}
DBUG_RETURN(0);
}
mi->mysql=0;
mi->file_id=1;
fn_format(fname, master_info_fname, mysql_data_home, "", 4+32);
/*
We need a mutex while we are changing master info parameters to
keep other threads from reading bogus info
*/
pthread_mutex_lock(&mi->data_lock);
fd = mi->fd;
/* does master.info exist ? */
if (access(fname,F_OK))
{
if (abort_if_no_master_info_file)
{
pthread_mutex_unlock(&mi->data_lock);
DBUG_RETURN(0);
}
/*
if someone removed the file from underneath our feet, just close
the old descriptor and re-create the old file
*/
if (fd >= 0)
my_close(fd, MYF(MY_WME));
if ((fd = my_open(fname, O_CREAT|O_RDWR|O_BINARY, MYF(MY_WME))) < 0 )
{
sql_print_error("Failed to create a new master info file (\
file '%s', errno %d)", fname, my_errno);
goto err;
}
if (init_io_cache(&mi->file, fd, IO_SIZE*2, READ_CACHE, 0L,0,
MYF(MY_WME)))
{
sql_print_error("Failed to create a cache on master info file (\
file '%s')", fname);
goto err;
}
mi->fd = fd;
init_master_info_with_options(mi);
}
else // file exists
{
if (fd >= 0)
reinit_io_cache(&mi->file, READ_CACHE, 0L,0,0);
else
{
if ((fd = my_open(fname, O_RDWR|O_BINARY, MYF(MY_WME))) < 0 )
{
sql_print_error("Failed to open the existing master info file (\
file '%s', errno %d)", fname, my_errno);
goto err;
}
if (init_io_cache(&mi->file, fd, IO_SIZE*2, READ_CACHE, 0L,
0, MYF(MY_WME)))
{
sql_print_error("Failed to create a cache on master info file (\
file '%s')", fname);
goto err;
}
}
mi->fd = fd;
int port, connect_retry, master_log_pos, lines;
int ssl= 0, ssl_verify_server_cert= 0;
char *first_non_digit;
/*
Starting from 4.1.x master.info has new format. Now its
first line contains number of lines in file. By reading this
number we will be always distinguish to which version our
master.info corresponds to. We can't simply count lines in
file since versions before 4.1.x could generate files with more
lines than needed.
If first line doesn't contain a number or contain number less than
LINES_IN_MASTER_INFO_WITH_SSL then such file is treated like file
from pre 4.1.1 version.
There is no ambiguity when reading an old master.info, as before
4.1.1, the first line contained the binlog's name, which is either
empty or has an extension (contains a '.'), so can't be confused
with an integer.
So we're just reading first line and trying to figure which version
is this.
*/
/*
The first row is temporarily stored in mi->master_log_name,
if it is line count and not binlog name (new format) it will be
overwritten by the second row later.
*/
if (init_strvar_from_file(mi->master_log_name,
sizeof(mi->master_log_name), &mi->file,
""))
goto errwithmsg;
lines= strtoul(mi->master_log_name, &first_non_digit, 10);
if (mi->master_log_name[0]!='\0' &&
*first_non_digit=='\0' && lines >= LINES_IN_MASTER_INFO_WITH_SSL)
{
/* Seems to be new format => read master log name from next line */
if (init_strvar_from_file(mi->master_log_name,
sizeof(mi->master_log_name), &mi->file, ""))
goto errwithmsg;
}
else
lines= 7;
if (init_intvar_from_file(&master_log_pos, &mi->file, 4) ||
init_strvar_from_file(mi->host, sizeof(mi->host), &mi->file,
master_host) ||
init_strvar_from_file(mi->user, sizeof(mi->user), &mi->file,
master_user) ||
init_strvar_from_file(mi->password, SCRAMBLED_PASSWORD_CHAR_LENGTH+1,
&mi->file, master_password) ||
init_intvar_from_file(&port, &mi->file, master_port) ||
init_intvar_from_file(&connect_retry, &mi->file,
master_connect_retry))
goto errwithmsg;
/*
If file has ssl part use it even if we have server without
SSL support. But these option will be ignored later when
slave will try connect to master, so in this case warning
is printed.
*/
if (lines >= LINES_IN_MASTER_INFO_WITH_SSL)
{
if (init_intvar_from_file(&ssl, &mi->file, master_ssl) ||
init_strvar_from_file(mi->ssl_ca, sizeof(mi->ssl_ca),
&mi->file, master_ssl_ca) ||
init_strvar_from_file(mi->ssl_capath, sizeof(mi->ssl_capath),
&mi->file, master_ssl_capath) ||
init_strvar_from_file(mi->ssl_cert, sizeof(mi->ssl_cert),
&mi->file, master_ssl_cert) ||
init_strvar_from_file(mi->ssl_cipher, sizeof(mi->ssl_cipher),
&mi->file, master_ssl_cipher) ||
init_strvar_from_file(mi->ssl_key, sizeof(mi->ssl_key),
&mi->file, master_ssl_key))
goto errwithmsg;
/*
Starting from 5.1.16 ssl_verify_server_cert might be
in the file
*/
if (lines >= LINE_FOR_MASTER_SSL_VERIFY_SERVER_CERT &&
init_intvar_from_file(&ssl_verify_server_cert, &mi->file, 0))
goto errwithmsg;
}
#ifndef HAVE_OPENSSL
if (ssl)
sql_print_warning("SSL information in the master info file "
"('%s') are ignored because this MySQL slave was compiled "
"without SSL support.", fname);
#endif /* HAVE_OPENSSL */
/*
This has to be handled here as init_intvar_from_file can't handle
my_off_t types
*/
mi->master_log_pos= (my_off_t) master_log_pos;
mi->port= (uint) port;
mi->connect_retry= (uint) connect_retry;
mi->ssl= (my_bool) ssl;
mi->ssl_verify_server_cert= ssl_verify_server_cert;
}
DBUG_PRINT("master_info",("log_file_name: %s position: %ld",
mi->master_log_name,
(ulong) mi->master_log_pos));
mi->rli.mi = mi;
if (init_relay_log_info(&mi->rli, slave_info_fname))
goto err;
mi->inited = 1;
// now change cache READ -> WRITE - must do this before flush_master_info
reinit_io_cache(&mi->file, WRITE_CACHE, 0L, 0, 1);
if ((error=test(flush_master_info(mi, 1))))
sql_print_error("Failed to flush master info file");
pthread_mutex_unlock(&mi->data_lock);
DBUG_RETURN(error);
errwithmsg:
sql_print_error("Error reading master configuration");
err:
if (fd >= 0)
{
my_close(fd, MYF(0));
end_io_cache(&mi->file);
}
mi->fd= -1;
pthread_mutex_unlock(&mi->data_lock);
DBUG_RETURN(1);
}
/*
RETURN
2 - flush relay log failed
1 - flush master info failed
0 - all ok
*/
int flush_master_info(Master_info* mi, bool flush_relay_log_cache)
{
IO_CACHE* file = &mi->file;
char lbuf[22];
DBUG_ENTER("flush_master_info");
DBUG_PRINT("enter",("master_pos: %ld", (long) mi->master_log_pos));
/*
Flush the relay log to disk. If we don't do it, then the relay log while
have some part (its last kilobytes) in memory only, so if the slave server
dies now, with, say, from master's position 100 to 150 in memory only (not
on disk), and with position 150 in master.info, then when the slave
restarts, the I/O thread will fetch binlogs from 150, so in the relay log
we will have "[0, 100] U [150, infinity[" and nobody will notice it, so the
SQL thread will jump from 100 to 150, and replication will silently break.
When we come to this place in code, relay log may or not be initialized;
the caller is responsible for setting 'flush_relay_log_cache' accordingly.
*/
if (flush_relay_log_cache &&
flush_io_cache(mi->rli.relay_log.get_log_file()))
DBUG_RETURN(2);
/*
We flushed the relay log BEFORE the master.info file, because if we crash
now, we will get a duplicate event in the relay log at restart. If we
flushed in the other order, we would get a hole in the relay log.
And duplicate is better than hole (with a duplicate, in later versions we
can add detection and scrap one event; with a hole there's nothing we can
do).
*/
/*
In certain cases this code may create master.info files that seems
corrupted, because of extra lines filled with garbage in the end
file (this happens if new contents take less space than previous
contents of file). But because of number of lines in the first line
of file we don't care about this garbage.
*/
my_b_seek(file, 0L);
my_b_printf(file,
"%u\n%s\n%s\n%s\n%s\n%s\n%d\n%d\n%d\n%s\n%s\n%s\n%s\n%s\n%d\n",
LINES_IN_MASTER_INFO,
mi->master_log_name, llstr(mi->master_log_pos, lbuf),
mi->host, mi->user,
mi->password, mi->port, mi->connect_retry,
(int)(mi->ssl), mi->ssl_ca, mi->ssl_capath, mi->ssl_cert,
mi->ssl_cipher, mi->ssl_key, mi->ssl_verify_server_cert);
DBUG_RETURN(-flush_io_cache(file));
}
void end_master_info(Master_info* mi)
{
DBUG_ENTER("end_master_info");
if (!mi->inited)
DBUG_VOID_RETURN;
end_relay_log_info(&mi->rli);
if (mi->fd >= 0)
{
end_io_cache(&mi->file);
(void)my_close(mi->fd, MYF(MY_WME));
mi->fd = -1;
}
mi->inited = 0;
DBUG_VOID_RETURN;
}
#endif /* HAVE_REPLICATION */
Reference in a new issue View git blame Copy permalink