mariadb/sql/strfunc.cc
Michael Widenius 1864d9596d Implementation of Multi-source replication (MDEV:253)
Documentation of the feature can be found at: http://kb.askmonty.org/en/multi-source-replication/
This code is based on code from Taobao, developed by Plinux

BUILD/SETUP.sh:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
client/mysqltest.cc:
  Added support for error names starting with 'W'
  Added connection_name support to --sync_with_master
cmake/maintainer.cmake:
  Added -Wno-invalid-offsetof to get rid of warning of offsetof() on C++ class (safe in the contex we use it)
mysql-test/r/mysqltest.result:
  Updated results
mysql-test/r/parser.result:
  Updated results
mysql-test/suite/multi_source/my.cnf:
  Setup of multi-master tests
mysql-test/suite/multi_source/simple.result:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/simple.test:
  Simple basic test of multi-source functionality
mysql-test/suite/multi_source/syntax.result:
  Test of multi-source syntax
mysql-test/suite/multi_source/syntax.test:
  Test of multi-source syntax
mysql-test/suite/rpl/r/rpl_rotate_logs.result:
  Updated results because of new error messages
mysql-test/t/parser.test:
  Updated test as master_pos_wait() now takes more arguments than before
sql/event_scheduler.cc:
  No reason to initialize slave_thread (it's guaranteed to be zero here)
sql/item_create.cc:
  Added connection_name argument to master_pos_wait()
  Simplified code
sql/item_func.cc:
  Added connection_name argument to master_pos_wait()
sql/item_func.h:
  Added connection_name argument to master_pos_wait()
sql/log.cc:
  Added tag "Master 'connection_name'" to slave errors that has a connection name.
sql/mysqld.cc:
  Added variable mysqld_server_initialized so that other functions can test if server is fully initialized.
  Free all slave data in one place (fewer ifdef's)
  Removed not needed call to close_active_mi()
  Initialize slaves() later in startup to ensure that everthing is really initialized when slaves start.
  Made status variable slave_running multi-source safe
sql/mysqld.h:
  Added mysqld_server_initialized
sql/rpl_mi.cc:
  Store connection name and cmp_connection_name (only used for show full slave status) in Master_info
  Added code for Master_info_index, which handles storage of multi-master information
  Don't write the empty "" connection_name to multi-master.info file. This is handled by the original code.
sql/rpl_mi.h:
  Added connection_name and Master_info_index
sql/rpl_rli.cc:
  Added connection_name to relay log files.
sql/rpl_rli.h:
  Fixed type of slave_skip_counter as we now access it directly in sys_vars.cc, so it must be uint
sql/share/errmsg-utf8.txt:
  Added new error messages needed for multi-source
  Added multi-source name to error ER_MASTER_INFO and WARN_NO_MASTER_INFO
sql/slave.cc:
  Moved things a bit around to make it easier to handle error conditions.
  Create a global master_info_index and add the "" connection to it
  Ensure that new Master_info doesn't fail.
  Don't call terminate_slave_threads(active_mi..) on end_slave() as this is now done automaticly when deleting master_info_index.
  Delete not needed function close_active_mi(). One can achive same thing by calling end_slave().
  Added support for SHOW FULL SLAVE STATUS (show status for all master connections with connection_name as first column)
sql/slave.h:
  Added new prototypes
sql/sql_base.cc:
  More DBUG_PRINT
sql/sql_class.cc:
  Reset thd->connection_name and thd-->default_master_connection
sql/sql_class.h:
  Added thd->connection_name and thd-->default_master_connection
  Added slave_skip_count to variables to make changing the @@sql_slave_skip_count variable thread safe
sql/sql_const.h:
  Added MAX_CONNECTION_NAME
sql/sql_lex.cc:
  Reset 'lex->verbose' (to simplify some sql_yacc.yy code)
sql/sql_lex.h:
  Added connection_name
sql/sql_parse.cc:
  Added support for connection_name to all SLAVE commands.
  - Instead of using active_mi, we now get the current Master_info from master_info_index.
  - Create new replication threads with CHANGE MASTER
  - Added support for show_all_master_info()
sql/sql_reload.cc:
  Made reset/full slave use master_info_index->get_master_info() instead of active_mi.
  If one uses 'RESET SLAVE "connection_name" all' the connection is removed from master_info_index.
sql/sql_repl.cc:
  sql_slave_skip_counter is moved to thd->variables to make it thread safe and fix some bugs with it
  Add connection name to relay log files.
  Added connection name to errors.
  Added some logging for multi-master if log_warnings > 1
  stop_slave():
  - Don't check if thd is set. It's guaranteed to always be set.
  change_master():
  - Check for duplicate connection names in change_master()
  - Check for wrong arguments first in file (to simplify error handling)
  - Register new connections in master_info_index
sql/sql_yacc.yy:
  Added optional connection_name to a all relevant master/slave commands
sql/strfunc.cc:
  my_global.h shoud always be included first.
sql/sys_vars.cc:
  Added variable default_master_connection
  Made variable sql_slave_skip_counter multi-source safe
sql/sys_vars.h:
  Added Sys_var_session_lexstring (needed for default_master_connection)
  Added Sys_var_multi_source_uint (needed for sql_slave_skip_counter).
2012-09-28 02:06:56 +03:00

407 lines
9.9 KiB
C++

/* Copyright (c) 2003, 2011, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
/* Some useful string utility functions used by the MySQL server */
#include <my_global.h>
#include "sql_priv.h"
#include "unireg.h"
#include "strfunc.h"
#include "sql_class.h"
#include "typelib.h" // TYPELIB
#include "m_ctype.h" // my_charset_latin1
#include "mysqld.h" // system_charset_info
/*
Return bitmap for strings used in a set
SYNOPSIS
find_set()
lib Strings in set
str Strings of set-strings separated by ','
err_pos If error, set to point to start of wrong set string
err_len If error, set to the length of wrong set string
set_warning Set to 1 if some string in set couldn't be used
NOTE
We delete all end space from str before comparison
RETURN
bitmap of all sets found in x.
set_warning is set to 1 if there was any sets that couldn't be set
*/
static const char field_separator=',';
ulonglong find_set(TYPELIB *lib, const char *str, uint length, CHARSET_INFO *cs,
char **err_pos, uint *err_len, bool *set_warning)
{
CHARSET_INFO *strip= cs ? cs : &my_charset_latin1;
const char *end= str + strip->cset->lengthsp(strip, str, length);
ulonglong found= 0;
*err_pos= 0; // No error yet
*err_len= 0;
if (str != end)
{
const char *start= str;
for (;;)
{
const char *pos= start;
uint var_len;
int mblen= 1;
if (cs && cs->mbminlen > 1)
{
for ( ; pos < end; pos+= mblen)
{
my_wc_t wc;
if ((mblen= cs->cset->mb_wc(cs, &wc, (const uchar *) pos,
(const uchar *) end)) < 1)
mblen= 1; // Not to hang on a wrong multibyte sequence
if (wc == (my_wc_t) field_separator)
break;
}
}
else
for (; pos != end && *pos != field_separator; pos++) ;
var_len= (uint) (pos - start);
uint find= cs ? find_type2(lib, start, var_len, cs) :
find_type(lib, start, var_len, (bool) 0);
if (!find && *err_len == 0) // report the first error with length > 0
{
*err_pos= (char*) start;
*err_len= var_len;
*set_warning= 1;
}
else
found|= ((longlong) 1 << (find - 1));
if (pos >= end)
break;
start= pos + mblen;
}
}
return found;
}
/*
Function to find a string in a TYPELIB
(similar to find_type() of mysys/typelib.c)
SYNOPSIS
find_type()
lib TYPELIB (struct of pointer to values + count)
find String to find
length Length of string to find
part_match Allow part matching of value
RETURN
0 error
> 0 position in TYPELIB->type_names +1
*/
uint find_type(const TYPELIB *lib, const char *find, uint length,
bool part_match)
{
uint found_count=0, found_pos=0;
const char *end= find+length;
const char *i;
const char *j;
for (uint pos=0 ; (j=lib->type_names[pos++]) ; )
{
for (i=find ; i != end &&
my_toupper(system_charset_info,*i) ==
my_toupper(system_charset_info,*j) ; i++, j++) ;
if (i == end)
{
if (! *j)
return(pos);
found_count++;
found_pos= pos;
}
}
return(found_count == 1 && part_match ? found_pos : 0);
}
/*
Find a string in a list of strings according to collation
SYNOPSIS
find_type2()
lib TYPELIB (struct of pointer to values + count)
x String to find
length String length
cs Character set + collation to use for comparison
NOTES
RETURN
0 No matching value
>0 Offset+1 in typelib for matched string
*/
uint find_type2(const TYPELIB *typelib, const char *x, uint length,
CHARSET_INFO *cs)
{
int pos;
const char *j;
DBUG_ENTER("find_type2");
DBUG_PRINT("enter",("x: '%.*s' lib: 0x%lx", length, x, (long) typelib));
if (!typelib->count)
{
DBUG_PRINT("exit",("no count"));
DBUG_RETURN(0);
}
for (pos=0 ; (j=typelib->type_names[pos]) ; pos++)
{
if (!my_strnncoll(cs, (const uchar*) x, length,
(const uchar*) j, typelib->type_lengths[pos]))
DBUG_RETURN(pos+1);
}
DBUG_PRINT("exit",("Couldn't find type"));
DBUG_RETURN(0);
} /* find_type */
/*
Un-hex all elements in a typelib
SYNOPSIS
unhex_type2()
interval TYPELIB (struct of pointer to values + lengths + count)
NOTES
RETURN
N/A
*/
void unhex_type2(TYPELIB *interval)
{
for (uint pos= 0; pos < interval->count; pos++)
{
char *from, *to;
for (from= to= (char*) interval->type_names[pos]; *from; )
{
/*
Note, hexchar_to_int(*from++) doesn't work
one some compilers, e.g. IRIX. Looks like a compiler
bug in inline functions in combination with arguments
that have a side effect. So, let's use from[0] and from[1]
and increment 'from' by two later.
*/
*to++= (char) (hexchar_to_int(from[0]) << 4) +
hexchar_to_int(from[1]);
from+= 2;
}
interval->type_lengths[pos] /= 2;
}
}
/*
Check if the first word in a string is one of the ones in TYPELIB
SYNOPSIS
check_word()
lib TYPELIB
val String to check
end End of input
end_of_word Store value of last used byte here if we found word
RETURN
0 No matching value
> 1 lib->type_names[#-1] matched
end_of_word will point to separator character/end in 'val'
*/
uint check_word(TYPELIB *lib, const char *val, const char *end,
const char **end_of_word)
{
int res;
const char *ptr;
/* Fiend end of word */
for (ptr= val ; ptr < end && my_isalpha(&my_charset_latin1, *ptr) ; ptr++)
;
if ((res=find_type(lib, val, (uint) (ptr - val), 1)) > 0)
*end_of_word= ptr;
return res;
}
/*
Converts a string between character sets
SYNOPSIS
strconvert()
from_cs source character set
from source, a null terminated string
to destination buffer
to_length destination buffer length
NOTES
'to' is always terminated with a '\0' character.
If there is no enough space to convert whole string,
only prefix is converted, and terminated with '\0'.
RETURN VALUES
result string length
*/
uint strconvert(CHARSET_INFO *from_cs, const char *from,
CHARSET_INFO *to_cs, char *to, uint to_length, uint *errors)
{
int cnvres;
my_wc_t wc;
char *to_start= to;
uchar *to_end= (uchar*) to + to_length - 1;
my_charset_conv_mb_wc mb_wc= from_cs->cset->mb_wc;
my_charset_conv_wc_mb wc_mb= to_cs->cset->wc_mb;
uint error_count= 0;
while (1)
{
/*
Using 'from + 10' is safe:
- it is enough to scan a single character in any character set.
- if remaining string is shorter than 10, then mb_wc will return
with error because of unexpected '\0' character.
*/
if ((cnvres= (*mb_wc)(from_cs, &wc,
(uchar*) from, (uchar*) from + 10)) > 0)
{
if (!wc)
break;
from+= cnvres;
}
else if (cnvres == MY_CS_ILSEQ)
{
error_count++;
from++;
wc= '?';
}
else
break; // Impossible char.
outp:
if ((cnvres= (*wc_mb)(to_cs, wc, (uchar*) to, to_end)) > 0)
to+= cnvres;
else if (cnvres == MY_CS_ILUNI && wc != '?')
{
error_count++;
wc= '?';
goto outp;
}
else
break;
}
*to= '\0';
*errors= error_count;
return (uint32) (to - to_start);
}
/*
Searches for a LEX_STRING in an LEX_STRING array.
SYNOPSIS
find_string_in_array()
heap The array
needle The string to search for
NOTE
The last LEX_STRING in the array should have str member set to NULL
RETURN VALUES
-1 Not found
>=0 Ordinal position
*/
int find_string_in_array(LEX_STRING * const haystack, LEX_STRING * const needle,
CHARSET_INFO * const cs)
{
const LEX_STRING *pos;
for (pos= haystack; pos->str; pos++)
if (!cs->coll->strnncollsp(cs, (uchar *) pos->str, pos->length,
(uchar *) needle->str, needle->length, 0))
{
return (pos - haystack);
}
return -1;
}
char *set_to_string(THD *thd, LEX_STRING *result, ulonglong set,
const char *lib[])
{
char buff[STRING_BUFFER_USUAL_SIZE*8];
String tmp(buff, sizeof(buff), &my_charset_latin1);
LEX_STRING unused;
if (!result)
result= &unused;
tmp.length(0);
for (uint i= 0; set; i++, set >>= 1)
if (set & 1) {
tmp.append(lib[i]);
tmp.append(',');
}
if (tmp.length())
{
result->str= thd->strmake(tmp.ptr(), tmp.length()-1);
result->length= tmp.length()-1;
}
else
{
result->str= const_cast<char*>("");
result->length= 0;
}
return result->str;
}
char *flagset_to_string(THD *thd, LEX_STRING *result, ulonglong set,
const char *lib[])
{
char buff[STRING_BUFFER_USUAL_SIZE*8];
String tmp(buff, sizeof(buff), &my_charset_latin1);
LEX_STRING unused;
if (!result) result= &unused;
tmp.length(0);
// note that the last element is always "default", and it's ignored below
for (uint i= 0; lib[i+1]; i++, set >>= 1)
{
tmp.append(lib[i]);
tmp.append(set & 1 ? "=on," : "=off,");
}
result->str= thd->strmake(tmp.ptr(), tmp.length()-1);
result->length= tmp.length()-1;
return result->str;
}