MDEV-35549 UBSAN: runtime error: applying zero offset to null pointer on XA RECOVER

With UBSAN builds the function my_string_repertoire_8bit() failed on
"runtime error: applying zero offset to null pointer" when
NULL wad passed as the str parameter.

Fix:

test str for NULL, and return MY_REPERTOIRE_ASCII if str is NULL.

MTR:

This problem made MTR tests
  - main.xa_sync
  - innodb.xa_debug
  - main.xa
fail with the nullptr-with-offset UNSAN error.
After this commit these tests do not fail anymore.
This commit does not need any new MTR tests.
This commit is contained in:
Alexander Barkov 2025-01-22 10:53:44 +04:00
commit aef6f35989

View file

@ -842,6 +842,8 @@ my_string_repertoire_8bit(CHARSET_INFO *cs, const char *str, size_t length)
const char *strend;
if ((cs->state & MY_CS_NONASCII) && length > 0)
return MY_REPERTOIRE_UNICODE30;
if (!str) // Avoid UBSAN nullptr-with-offset
return MY_REPERTOIRE_ASCII;
for (strend= str + length; str < strend; str++)
{
if (((uchar) *str) > 0x7F)