mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-04-12 02:05:34 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
mariadb/strings/ctype-uca0900.c
Alexander Barkov 1748b02ec6 MDEV-36213 Doubled memory usage (11.4.4 <-> 11.4.5) 
Fixing the code adding MySQL _0900_ collations as _uca1400_ aliases
not to perform deep initialization of the corresponding _uca1400_
collations.

Only basic initialization is now performed which allows to watch
these collations (both _0900_ and _uca1400_) in queries to
INFORMATION_SCHEMA tables COLLATIONS and
COLLATION_CHARACTER_SET_APPLICABILITY,
as well as in SHOW COLLATION statements.

Deep initialization is now performed only when a collation
(either the _0900_ alias or the corresponding  _uca1400_ collation)
is used for the very first time after the server startup.

Refactoring was done to maintain the code easier:
- most of the _uca1400_ code was moved from ctype-uca.c
  to a new file ctype-uca1400.c
- most of the _0900_ code was moved from type-uca.c
  to a new file ctype-uca0900.c

Change details:

- The original function add_alias_for_collation() added by the patch for
   "MDEV-20912 Add support for utf8mb4_0900_* collations in MariaDB Server"
  was removed from mysys/charset.c, as it had two two problems:

  a. it forced deep initialization of the _uca1400_ collations
     when adding _0900_ aliases for them at the server startup
     (the main reported problem)

  b. it introduced cyclic dependency between /mysys and /strings
     - /mysys/charset-def.c depended on /strings/ctype-uca.c
     - /strings/ctype-uca.c depended on /mysys/charset.c

  The code from add_alias_for_collation() was split into separate functions.
  Cyclic dependency was removed. `#include <my_sys.h>` was removed
  from /strings/ctype-uca.c. Collations are now added using a callback
  function MY_CHARSET_LOADED::add_collation, like it is done for
  user collations defined in Index.xml. The code in /mysys sets
  MY_CHARSET_LOADED::add_collation to add_compiled_collation().

- The function compare_collations() was removed.
  A new virtual function was added into my_collation_handler_st instead:

    my_bool (*eq_collation)(CHARSET_INFO *self, CHARSET_INFO *other);

  because it is the collation handler who knows how to detect equal
  collations by comparing only some of CHARSET_INFO members without
  their deep initialization.

  Three implementations were added:
  - my_ci_eq_collation_uca() for UCA collations, it compares
    _0900_ collations as equal to their corresponding _uca1400_ collations.
  - my_ci_eq_collation_utf8mb4_bin(), it compares
    utf8mb4_nopad_bin and utf8mb4_0900_bin as equal.
  - my_ci_eq_collation_generic() - the default implementation,
    which compares all collations as not equal.

  A C++ wrapper CHARSET_INFO::eq_collations() was added.
  The code in /sql was changes to use the wrapper instead of
  the former calls for the removed function compare_collations().

- A part of add_alias_for_collation() was moved into a new function
  my_ci_alloc(). It allocates a memory for a new charset_info_st
  instance together with the collation name and the comment using a single
  MY_CHARSET_LOADER::once_alloc call, which normally points to my_once_alloc().

- A part of add_alias_for_collation() was moved into a new function
  my_ci_make_comment_for_alias(). It makes an "Alias for xxx" string,
  e.g. "Alias for utf8mb4_uca1400_swedish_ai_ci" in case of
  utf8mb4_sv_0900_ai_ci.

- A part of the code in create_tailoring() was moved to
  a new function my_uca1400_collation_get_initialized_shared_uca(),
  to reuse the code between _uca1400_ and _0900_ collations.

- A new function my_collation_id_is_mysql_uca0900() was added
  in addition to my_collation_id_is_mysql_uca1400().

- Functions to build collation names were added:
   my_uca0900_collation_build_name()
   my_uca1400_collation_build_name()

- A shared function function was added:

  my_bool
  my_uca1400_collation_alloc_and_init(MY_CHARSET_LOADER *loader,
                                      LEX_CSTRING name,
                                      LEX_CSTRING comment,
                                      const uca_collation_def_param_t *param,
                                      uint id)

  It's reused to add _uca1400_ and _0900_ collations, with basic
  initialization (without deep initialization).

- The function add_compiled_collation() changed its return type from
  void to int, to make it compatible with MY_CHARSET_LOADER::add_collation.

- Functions mysql_uca0900_collation_definition_add(),
  mysql_uca0900_utf8mb4_collation_definitions_add(),
  mysql_utf8mb4_0900_bin_add() were added into ctype-uca0900.c.
  They get MY_CHARSET_LOADER as a parameter.

- Functions my_uca1400_collation_definition_add(),
  my_uca1400_collation_definitions_add() were moved from
  charset-def.c to strings/ctype-uca1400.c.
  The latter now accepts MY_CHARSET_LOADER as the first parameter
  instead of initializing a MY_CHARSET_LOADER inside.

- init_compiled_charsets() now initializes a MY_CHARSET_LOADER
  variable and passes it to all functions adding collations:
  - mysql_utf8mb4_0900_collation_definitions_add()
  - mysql_uca0900_utf8mb4_collation_definitions_add()
  - mysql_utf8mb4_0900_bin_add()

- A new structure was added into ctype-uca.h:

  typedef struct uca_collation_def_param
  {
    my_cs_encoding_t cs_id;
    uint tailoring_id;
    uint nopad_flags;
    uint level_flags;
  } uca_collation_def_param_t;

  It simplifies reusing the code for _uca1400_ and _0900_ collations.

- The definition of MY_UCA1400_COLLATION_DEFINITION was
  moved from ctype-uca.c to ctype-uca1400.h, to reuse
  the code for _uca1400_ and _0900_ collations.

- The definitions of "MY_UCA_INFO my_uca_v1400" and
  "MY_UCA_INFO my_uca1400_info_tailored[][]" were moved from
  ctype-uca.c to ctype-uca1400.c.

- The definitions/declarations of:
  - mysql_0900_collation_start,
  - struct mysql_0900_to_mariadb_1400_mapping
  - mysql_0900_to_mariadb_1400_mapping
  - mysql_utf8mb4_0900_collation_definitions_add()
  were moved from ctype-uca.c to ctype-uca0900.c

- Functions
  my_uca1400_make_builtin_collation_id()
  my_uca1400_collation_definition_init()
  my_uca1400_collation_id_uca400_compat()
  my_ci_get_collation_name_uca1400_context()
  were moved from ctype-uca.c to ctype-uca1400.c and ctype-uca1400.h

- A part of my_uca1400_collation_definition_init()
  was moved into my_uca1400_collation_source(),
  to make functions smaller.
2025-03-31 18:17:26 +04:00

209 lines
7.5 KiB
C
Raw Permalink Blame History

/* Copyright (c) 2025, MariaDB
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Library General Public
License as published by the Free Software Foundation; version 2
of the License.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Library General Public License for more details.
You should have received a copy of the GNU Library General Public
License along with this library; if not, write to the Free
Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston,
MA 02110-1335 USA */
#include "my_global.h"
#include "strings_def.h"
#include "ctype-uca.h"
struct mysql_0900_to_mariadb_1400_mapping
mysql_0900_mapping[mysql_0900_collation_num]=
{
/* 255 Ascent insensitive, Case insensitive 'ai_ci' */
{"", "", "ai_ci", 2308},
{"de_pb", "german2", "ai_ci", 2468},
{"is", "icelandic", "ai_ci", 2316},
{"lv", "latvian", "ai_ci", 2324},
{"ro", "romanian", "ai_ci", 2332},
{"sl", "slovenian", "ai_ci", 2340},
{"pl", "polish", "ai_ci", 2348},
{"et", "estonian", "ai_ci", 2356},
{"es", "spanish", "ai_ci", 2364},
{"sv", "swedish", "ai_ci", 2372},
{"tr", "turkish", "ai_ci", 2380},
{"cs", "czech", "ai_ci", 2388},
{"da", "danish", "ai_ci", 2396},
{"lt", "lithuanian", "ai_ci", 2404},
{"sk", "slovak", "ai_ci", 2412},
{"es_trad", "spanish2", "ai_ci", 2420},
{"la", "roman", "ai_ci", 2428},
{"fa", NullS, "ai_ci", 0}, // Disabled in MySQL
{"eo", "esperanto", "ai_ci", 2444},
{"hu", "hungarian", "ai_ci", 2452},
{"hr", "croatian", "ai_ci", 2500},
{"si", NullS, "ai_ci", 0}, // Disabled in MySQL
{"vi", "vietnamese", "ai_ci", 2492},
/* 278 Ascent sensitive, Case sensitive 'as_cs' */
{"","", "as_cs", 2311},
{"de_pb", "german2", "as_cs", 2471},
{"is", "icelandic", "as_cs", 2319},
{"lv", "latvian", "as_cs", 2327},
{"ro", "romanian", "as_cs", 2335},
{"sl", "slovenian", "as_cs", 2343},
{"pl", "polish", "as_cs", 2351},
{"et", "estonian", "as_cs", 2359},
{"es", "spanish", "as_cs", 2367},
{"sv", "swedish", "as_cs", 2375},
{"tr", "turkish", "as_cs", 2383},
{"cs", "czech", "as_cs", 2391},
{"da", "danish", "as_cs", 2399},
{"lt", "lithuanian", "as_cs", 2407},
{"sk", "slovak", "as_cs", 2415},
{"es_trad", "spanish2", "as_cs", 2423},
{"la", "roman", "as_cs", 2431},
{"fa", NullS, "as_cs", 0}, // Disabled in MySQL
{"eo", "esperanto", "as_cs", 2447},
{"hu", "hungarian", "as_cs", 2455},
{"hr", "croatian", "as_cs", 2503},
{"si", NullS, "as_cs", 0}, // Disabled in MySQL
{"vi", "vietnamese", "as_cs", 2495},
{"", NullS, "as_cs", 0}, // Missing
{"", NullS, "as_cs", 0}, // Missing
{"_ja_0900_as_cs", NullS, "as_cs", 0}, // Not supported
{"_ja_0900_as_cs_ks", NullS, "as_cs", 0}, // Not supported
/* 305 Ascent-sensitive, Case insensitive 'as_ci' */
{"","", "as_ci", 2310},
{"ru", NullS, "ai_ci", 0}, // Not supported
{"ru", NullS, "as_cs", 0}, // Not supported
{"zh", NullS, "as_cs", 0}, // Not supported
{NullS, NullS, "", 0}
};
static LEX_CSTRING
my_uca0900_collation_build_name(char *buffer, size_t buffer_size,
const char *cs_name,
const char *tailoring_name,
const char *sensitivity_suffix)
{
LEX_CSTRING res;
DBUG_ASSERT(buffer_size > 1);
res.str= buffer;
res.length= (strxnmov(buffer, buffer_size - 1,
cs_name, "_", tailoring_name,
(tailoring_name[0] ? "_" : ""),
"0900_",
sensitivity_suffix,
NullS) - buffer);
return res;
}
/*
Add a MySQL UCA-0900 collation as an alias for a MariaDB UCA-1400 collation.
*/
static my_bool
mysql_uca0900_collation_definition_add(MY_CHARSET_LOADER *loader,
const struct mysql_0900_to_mariadb_1400_mapping *map,
uint alias_id)
{
char comment_buffer[MY_CS_COLLATION_NAME_SIZE + 15];
char alias_buffer[MY_CS_COLLATION_NAME_SIZE + 1];
char name1400_buffer[MY_CS_COLLATION_NAME_SIZE + 1];
LEX_CSTRING comment= {comment_buffer, 0};
LEX_CSTRING alias_name= {alias_buffer, 0};
LEX_CSTRING name1400= {name1400_buffer, 0};
LEX_CSTRING utf8mb4= {STRING_WITH_LEN("utf8mb4")};
uint offset_in_map= alias_id - mysql_0900_collation_start;
uint id1400= mysql_0900_mapping[offset_in_map].collation_id;
uca_collation_def_param_t param= my_uca1400_collation_param_by_id(id1400);
const MY_UCA1400_COLLATION_DEFINITION *def1400=
&my_uca1400_collation_definitions[param.tailoring_id];
DBUG_ASSERT(my_collation_id_is_mysql_uca0900(alias_id));
alias_name= my_uca0900_collation_build_name(alias_buffer,
sizeof(alias_buffer),
"utf8mb4",
map->mysql_col_name,
map->case_sensitivity);
name1400= my_uca1400_collation_build_name(name1400_buffer,
sizeof(name1400_buffer),
&utf8mb4, def1400->name, &param);
comment= my_ci_make_comment_for_alias(comment_buffer, sizeof(comment_buffer),
name1400.str);
#ifdef DEBUG_PRINT_ALIAS
fprintf(stderr, "alias[%u] %-26s -> [%u] %s\n",
id, alias_name.str, id1400, name1400.str);
#endif
return my_uca1400_collation_alloc_and_init(loader, alias_name,
comment, &param, alias_id);
}
/*
Add support for MySQL 8.0 utf8mb4_0900_.. UCA collations.
The collation id's were collected from fprintf()
in mysql_uca0900_collation_definition_add().
Map mysql character sets to MariaDB using the same definition but
with the MySQL collation name and id.
*/
my_bool
mysql_uca0900_utf8mb4_collation_definitions_add(MY_CHARSET_LOADER *loader)
{
uint alias_id= mysql_0900_collation_start;
struct mysql_0900_to_mariadb_1400_mapping *map;
for (map= mysql_0900_mapping; map->mysql_col_name ; map++, alias_id++)
{
if (map->mariadb_col_name) /* Supported collation */
{
if (mysql_uca0900_collation_definition_add(loader, map, alias_id))
return TRUE;
}
}
return FALSE;
}
/*
Add MySQL utf8mb4_0900_bin collation as
an alias for MariaDB utf8mb4_nopad_bin.
*/
my_bool mysql_utf8mb4_0900_bin_add(MY_CHARSET_LOADER *loader)
{
CHARSET_INFO *src= &my_charset_utf8mb4_nopad_bin;
LEX_CSTRING alias_name= {STRING_WITH_LEN("utf8mb4_0900_bin")};
uint alias_id= 309;
char comment_buffer[MY_CS_COLLATION_NAME_SIZE+15];
LEX_CSTRING comment= my_ci_make_comment_for_alias(comment_buffer,
sizeof(comment_buffer),
src->coll_name.str);
struct charset_info_st *dst= my_ci_alloc(loader, alias_name, &alias_name,
comment, &comment);
if (!dst)
return TRUE;
*dst= *src;
dst->number= alias_id;
dst->coll_name= alias_name;
dst->comment= comment.str;
(loader->add_collation)(dst);
return FALSE;
}
Reference in a new issue View git blame Copy permalink