mirror of
https://github.com/MariaDB/server.git
synced 2025-04-11 17:55:34 +02:00
1748b02ec6
Fixing the code adding MySQL _0900_ collations as _uca1400_ aliases
not to perform deep initialization of the corresponding _uca1400_
collations.
Only basic initialization is now performed which allows to watch
these collations (both _0900_ and _uca1400_) in queries to
INFORMATION_SCHEMA tables COLLATIONS and
COLLATION_CHARACTER_SET_APPLICABILITY,
as well as in SHOW COLLATION statements.
Deep initialization is now performed only when a collation
(either the _0900_ alias or the corresponding _uca1400_ collation)
is used for the very first time after the server startup.
Refactoring was done to maintain the code easier:
- most of the _uca1400_ code was moved from ctype-uca.c
to a new file ctype-uca1400.c
- most of the _0900_ code was moved from type-uca.c
to a new file ctype-uca0900.c
Change details:
- The original function add_alias_for_collation() added by the patch for
"MDEV-20912 Add support for utf8mb4_0900_* collations in MariaDB Server"
was removed from mysys/charset.c, as it had two two problems:
a. it forced deep initialization of the _uca1400_ collations
when adding _0900_ aliases for them at the server startup
(the main reported problem)
b. it introduced cyclic dependency between /mysys and /strings
- /mysys/charset-def.c depended on /strings/ctype-uca.c
- /strings/ctype-uca.c depended on /mysys/charset.c
The code from add_alias_for_collation() was split into separate functions.
Cyclic dependency was removed. `#include <my_sys.h>` was removed
from /strings/ctype-uca.c. Collations are now added using a callback
function MY_CHARSET_LOADED::add_collation, like it is done for
user collations defined in Index.xml. The code in /mysys sets
MY_CHARSET_LOADED::add_collation to add_compiled_collation().
- The function compare_collations() was removed.
A new virtual function was added into my_collation_handler_st instead:
my_bool (*eq_collation)(CHARSET_INFO *self, CHARSET_INFO *other);
because it is the collation handler who knows how to detect equal
collations by comparing only some of CHARSET_INFO members without
their deep initialization.
Three implementations were added:
- my_ci_eq_collation_uca() for UCA collations, it compares
_0900_ collations as equal to their corresponding _uca1400_ collations.
- my_ci_eq_collation_utf8mb4_bin(), it compares
utf8mb4_nopad_bin and utf8mb4_0900_bin as equal.
- my_ci_eq_collation_generic() - the default implementation,
which compares all collations as not equal.
A C++ wrapper CHARSET_INFO::eq_collations() was added.
The code in /sql was changes to use the wrapper instead of
the former calls for the removed function compare_collations().
- A part of add_alias_for_collation() was moved into a new function
my_ci_alloc(). It allocates a memory for a new charset_info_st
instance together with the collation name and the comment using a single
MY_CHARSET_LOADER::once_alloc call, which normally points to my_once_alloc().
- A part of add_alias_for_collation() was moved into a new function
my_ci_make_comment_for_alias(). It makes an "Alias for xxx" string,
e.g. "Alias for utf8mb4_uca1400_swedish_ai_ci" in case of
utf8mb4_sv_0900_ai_ci.
- A part of the code in create_tailoring() was moved to
a new function my_uca1400_collation_get_initialized_shared_uca(),
to reuse the code between _uca1400_ and _0900_ collations.
- A new function my_collation_id_is_mysql_uca0900() was added
in addition to my_collation_id_is_mysql_uca1400().
- Functions to build collation names were added:
my_uca0900_collation_build_name()
my_uca1400_collation_build_name()
- A shared function function was added:
my_bool
my_uca1400_collation_alloc_and_init(MY_CHARSET_LOADER *loader,
LEX_CSTRING name,
LEX_CSTRING comment,
const uca_collation_def_param_t *param,
uint id)
It's reused to add _uca1400_ and _0900_ collations, with basic
initialization (without deep initialization).
- The function add_compiled_collation() changed its return type from
void to int, to make it compatible with MY_CHARSET_LOADER::add_collation.
- Functions mysql_uca0900_collation_definition_add(),
mysql_uca0900_utf8mb4_collation_definitions_add(),
mysql_utf8mb4_0900_bin_add() were added into ctype-uca0900.c.
They get MY_CHARSET_LOADER as a parameter.
- Functions my_uca1400_collation_definition_add(),
my_uca1400_collation_definitions_add() were moved from
charset-def.c to strings/ctype-uca1400.c.
The latter now accepts MY_CHARSET_LOADER as the first parameter
instead of initializing a MY_CHARSET_LOADER inside.
- init_compiled_charsets() now initializes a MY_CHARSET_LOADER
variable and passes it to all functions adding collations:
- mysql_utf8mb4_0900_collation_definitions_add()
- mysql_uca0900_utf8mb4_collation_definitions_add()
- mysql_utf8mb4_0900_bin_add()
- A new structure was added into ctype-uca.h:
typedef struct uca_collation_def_param
{
my_cs_encoding_t cs_id;
uint tailoring_id;
uint nopad_flags;
uint level_flags;
} uca_collation_def_param_t;
It simplifies reusing the code for _uca1400_ and _0900_ collations.
- The definition of MY_UCA1400_COLLATION_DEFINITION was
moved from ctype-uca.c to ctype-uca1400.h, to reuse
the code for _uca1400_ and _0900_ collations.
- The definitions of "MY_UCA_INFO my_uca_v1400" and
"MY_UCA_INFO my_uca1400_info_tailored[][]" were moved from
ctype-uca.c to ctype-uca1400.c.
- The definitions/declarations of:
- mysql_0900_collation_start,
- struct mysql_0900_to_mariadb_1400_mapping
- mysql_0900_to_mariadb_1400_mapping
- mysql_utf8mb4_0900_collation_definitions_add()
were moved from ctype-uca.c to ctype-uca0900.c
- Functions
my_uca1400_make_builtin_collation_id()
my_uca1400_collation_definition_init()
my_uca1400_collation_id_uca400_compat()
my_ci_get_collation_name_uca1400_context()
were moved from ctype-uca.c to ctype-uca1400.c and ctype-uca1400.h
- A part of my_uca1400_collation_definition_init()
was moved into my_uca1400_collation_source(),
to make functions smaller.
177 lines
6.5 KiB
C
177 lines
6.5 KiB
C
#ifndef STRINGS_DEF_INCLUDED
|
|
#define STRINGS_DEF_INCLUDED
|
|
/* Copyright (C) 2011 Monty Program Ab
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
it under the terms of the GNU General Public License as published by
|
|
the Free Software Foundation; version 2 of the License.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with this program; if not, write to the Free Software
|
|
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335 USA */
|
|
|
|
/* This file is to be include first in all files in the string directory */
|
|
|
|
#undef DBUG_ASSERT_AS_PRINTF
|
|
#include <my_global.h> /* Define standard vars */
|
|
#include "m_string.h" /* Exernal definitions of string functions */
|
|
#include "m_ctype.h"
|
|
|
|
/*
|
|
We can't use the original DBUG_ASSERT() (which includes _db_flush())
|
|
in the strings library as libdbug is compiled after the the strings
|
|
library and we don't want to have strings depending on libdbug which
|
|
depends on mysys and strings.
|
|
*/
|
|
|
|
#if !defined(DBUG_OFF)
|
|
#undef DBUG_ASSERT
|
|
#define DBUG_ASSERT(A) assert(A)
|
|
#endif
|
|
|
|
#define MY_NOPAD_ID(x) ((x)+0x400)
|
|
|
|
/* SPACE_INT is a word that contains only spaces */
|
|
#if SIZEOF_INT == 4
|
|
#define SPACE_INT 0x20202020
|
|
#elif SIZEOF_INT == 8
|
|
#define SPACE_INT 0x2020202020202020
|
|
#else
|
|
#error define the appropriate constant for a word full of spaces
|
|
#endif
|
|
|
|
/**
|
|
Skip trailing space.
|
|
|
|
On most systems reading memory in larger chunks (ideally equal to the size of
|
|
the chinks that the machine physically reads from memory) causes fewer memory
|
|
access loops and hence increased performance.
|
|
This is why the 'int' type is used : it's closest to that (according to how
|
|
it's defined in C).
|
|
So when we determine the amount of whitespace at the end of a string we do
|
|
the following :
|
|
1. We divide the string into 3 zones :
|
|
a) from the start of the string (__start) to the first multiple
|
|
of sizeof(int) (__start_words)
|
|
b) from the end of the string (__end) to the last multiple of sizeof(int)
|
|
(__end_words)
|
|
c) a zone that is aligned to sizeof(int) and can be safely accessed
|
|
through an int *
|
|
2. We start comparing backwards from (c) char-by-char. If all we find is
|
|
space then we continue
|
|
3. If there are elements in zone (b) we compare them as unsigned ints to a
|
|
int mask (SPACE_INT) consisting of all spaces
|
|
4. Finally we compare the remaining part (a) of the string char by char.
|
|
This covers for the last non-space unsigned int from 3. (if any)
|
|
|
|
This algorithm works well for relatively larger strings, but it will slow
|
|
the things down for smaller strings (because of the additional calculations
|
|
and checks compared to the naive method). Thus the barrier of length 20
|
|
is added.
|
|
|
|
@param ptr pointer to the input string
|
|
@param len the length of the string
|
|
@return the last non-space character
|
|
*/
|
|
|
|
static inline const uchar *skip_trailing_space(const uchar *ptr,size_t len)
|
|
{
|
|
const uchar *end= ptr + len;
|
|
DBUG_ASSERT(ptr); /* Avoid UBSAN nullptr-with-offset */
|
|
if (len > 20)
|
|
{
|
|
const uchar *end_words= (const uchar *)(intptr)
|
|
(((ulonglong)(intptr)end) / SIZEOF_INT * SIZEOF_INT);
|
|
const uchar *start_words= (const uchar *)(intptr)
|
|
((((ulonglong)(intptr)ptr) + SIZEOF_INT - 1) / SIZEOF_INT * SIZEOF_INT);
|
|
|
|
DBUG_ASSERT(((ulonglong)(intptr)ptr) >= SIZEOF_INT);
|
|
if (end_words > ptr)
|
|
{
|
|
while (end > end_words && end[-1] == 0x20)
|
|
end--;
|
|
if (end[-1] == 0x20 && start_words < end_words)
|
|
while (end > start_words && ((unsigned *)end)[-1] == SPACE_INT)
|
|
end -= SIZEOF_INT;
|
|
}
|
|
}
|
|
while (end > ptr && end[-1] == 0x20)
|
|
end--;
|
|
return (end);
|
|
}
|
|
|
|
|
|
int my_strnncollsp_nchars_generic(CHARSET_INFO *cs,
|
|
const uchar *str1, size_t len1,
|
|
const uchar *str2, size_t len2,
|
|
size_t nchars,
|
|
uint flags);
|
|
|
|
int my_strnncollsp_nchars_generic_8bit(CHARSET_INFO *cs,
|
|
const uchar *str1, size_t len1,
|
|
const uchar *str2, size_t len2,
|
|
size_t nchars,
|
|
uint flags);
|
|
|
|
uint my_8bit_charset_flags_from_data(CHARSET_INFO *cs);
|
|
uint my_8bit_collation_flags_from_data(CHARSET_INFO *cs);
|
|
|
|
|
|
/* Macros for hashing characters */
|
|
|
|
#define MY_HASH_ADD(A, B, value) \
|
|
do { A^= (((A & 63)+B)*((value)))+ (A << 8); B+=3; } while(0)
|
|
|
|
#define MY_HASH_ADD_16(A, B, value) \
|
|
do { MY_HASH_ADD(A, B, ((value) & 0xFF)) ; MY_HASH_ADD(A, B, ((value >>8 ))); } while(0)
|
|
|
|
|
|
#define my_wc_t ulong
|
|
|
|
int my_wc_to_printable_ex(CHARSET_INFO *cs, my_wc_t wc,
|
|
uchar *s, uchar *e,
|
|
uint bs, uint bslen, uint diglen);
|
|
|
|
int my_wc_to_printable_generic(CHARSET_INFO *cs, my_wc_t wc,
|
|
uchar *s, uchar *e);
|
|
|
|
int my_wc_to_printable_8bit(CHARSET_INFO *cs, my_wc_t wc,
|
|
uchar *s, uchar *e);
|
|
|
|
void my_ci_set_strength(struct charset_info_st *cs, uint strength);
|
|
void my_ci_set_level_flags(struct charset_info_st *cs, uint flags);
|
|
|
|
uint my_casefold_multiply_1(CHARSET_INFO *cs);
|
|
uint my_casefold_multiply_2(CHARSET_INFO *cs);
|
|
|
|
my_bool my_ci_eq_collation_generic(CHARSET_INFO *self, CHARSET_INFO *other);
|
|
|
|
LEX_CSTRING my_ci_make_comment_for_alias(char *buffer, size_t buffer_size,
|
|
const char *srcname);
|
|
|
|
struct charset_info_st *my_ci_alloc(MY_CHARSET_LOADER *loader,
|
|
const LEX_CSTRING name,
|
|
LEX_CSTRING *out_name,
|
|
const LEX_CSTRING comment,
|
|
LEX_CSTRING *out_comment);
|
|
|
|
/* Some common character set names */
|
|
extern const char charset_name_latin2[];
|
|
#define charset_name_latin2_length 6
|
|
extern const char charset_name_utf8mb3[];
|
|
#define charset_name_utf8mb3_length 7
|
|
extern const char charset_name_utf16[];
|
|
#define charset_name_utf16_length 5
|
|
extern const char charset_name_utf32[];
|
|
#define charset_name_utf32_length 5
|
|
extern const char charset_name_ucs2[];
|
|
#define charset_name_ucs2_length 4
|
|
extern const char charset_name_utf8mb4[];
|
|
#define charset_name_utf8mb4_length 7
|
|
|
|
#endif /*STRINGS_DEF_INCLUDED */
|