Commit graph

291 commits

Author SHA1 Message Date
Marko Mäkelä
03357ded17 Merge 10.4 into 10.5 2020-10-30 13:53:10 +02:00
Marko Mäkelä
cb253b8687 MDEV-22387: Static_binary_string::q_append() invokes memcpy on NULL
Invoking memcpy() on a NULL pointer is undefined behaviour
(even if the length is 0) and gives the compiler permission to
assume that the pointer is nonnull. Recent versions of GCC
(starting with version 8) are more aggressively optimizing away
checks for NULL pointers. This undefined behaviour would cause
a SIGSEGV in the test main.func_encrypt on an optimized debug build
on GCC 10.2.0.
2020-10-30 13:07:42 +02:00
Vicențiu Ciorbaru
85c686e2d1 cleanup: Static_binary_string need not take non-const double parameter
Convert the parameter to const as the function won't modify the pointer
value.
2020-10-28 11:38:14 +02:00
Marko Mäkelä
1c58748196 Merge 10.4 into 10.5 2020-08-10 21:38:55 +03:00
Marko Mäkelä
101ddc5e27 Merge mariadb-10.4.14 2020-08-10 20:37:52 +03:00
Alexander Barkov
fe555b9c5f MDEV-23415 Server crash or Assertion `dec_length <= str_length' failed in Item_func_format::val_str_ascii
Problem:

The crash happened in FORMAT(double, dec>=31, 'de_DE').

The patch for MDEV-23118 (commit 0041dacc1b)
did not take into account that String::set_real() has a limit of 31
(FLOATING_POINT_DECIMALS) fractional digits. So for the range of 31..38
digits, set_real() switches to use:
- my_fcvt() - decimal point notation, e.g. 1.9999999999
- my_gcvt() - scientific notation,    e.g. 1e22

my_gcvt() returned a shorter string than Item_func_format::val_str_ascii()
expected to get after the my_fcvt() call, so it crashed on assert.

Solution:

We cannot extend set_real() to use the my_fcvt() mode for the range of
31..38 fractional digits, because set_real() is used in a lot of places
and such a change will break everything.

Introducing String::set_fcvt() which always prints using my_fcvt()
for the whole range of decimals 0..38, supported by the FORMAT() function.
2020-08-08 09:44:31 +04:00
Sergei Golubchik
cd2924bacb MDEV-23330 Server crash or ASAN negative-size-param in my_strnncollsp_binary / SORT_FIELD_ATTR::compare_packed_varstrings
and
MDEV-23414 Assertion `res->charset() == item->collation.collation' failed in Type_handler_string_result::make_packed_sort_key_part

pack_sort_string() *must* take a collation from the Item, not from the
String value. Because when casting a string to _binary the original
String is not copied for performance reasons, it's reused but its
collation does not match Item's collation anymore.

Note, that String's collation cannot be simply changed to _binary,
because for an Item_string literal the original String must stay
unchanged for the duration of the query.

this partially reverts 61c15ebe32
2020-08-07 13:39:04 +02:00
Oleksandr Byelkin
57325e4706 Merge branch '10.3' into 10.4 2020-08-03 14:44:06 +02:00
Alexander Barkov
d63631c3fa MDEV-19632 Replication aborts with ER_SLAVE_CONVERSION_FAILED upon CREATE ... SELECT in ORACLE mode
- Adding optional qualifiers to data types:
    CREATE TABLE t1 (a schema.DATE);
  Qualifiers now work only for three pre-defined schemas:

    mariadb_schema
    oracle_schema
    maxdb_schema

  These schemas are virtual (hard-coded) for now, but may turn into real
  databases on disk in the future.

- mariadb_schema.TYPE now always resolves to a true MariaDB data
  type TYPE without sql_mode specific translations.

- oracle_schema.DATE translates to MariaDB DATETIME.

- maxdb_schema.TIMESTAMP translates to MariaDB DATETIME.

- Fixing SHOW CREATE TABLE to use a qualifier for a data type TYPE
  if the current sql_mode translates TYPE to something else.

The above changes fix the reported problem, so this script:

    SET sql_mode=ORACLE;
    CREATE TABLE t2 AS SELECT mariadb_date_column FROM t1;

is now replicated as:

    SET sql_mode=ORACLE;
    CREATE TABLE t2 (mariadb_date_column mariadb_schema.DATE);

and the slave can unambiguously treat DATE as the true MariaDB DATE
without ORACLE specific translation to DATETIME.

Similar,

    SET sql_mode=MAXDB;
    CREATE TABLE t2 AS SELECT mariadb_timestamp_column FROM t1;

is now replicated as:

    SET sql_mode=MAXDB;
    CREATE TABLE t2 (mariadb_timestamp_column mariadb_schema.TIMESTAMP);

so the slave treats TIMESTAMP as the true MariaDB TIMESTAMP
without MAXDB specific translation to DATETIME.
2020-08-01 07:43:50 +04:00
Monty
61c15ebe32 Remove String::lex_string() and String::lex_cstring()
- Better to use 'String *' directly.
- Added String::get_value(LEX_STRING*) for the few cases where we want to
  convert a String to LEX_CSTRING.

Other things:
- Use StringBuffer for some functions to avoid mallocs
2020-07-23 10:54:32 +03:00
Marko Mäkelä
fbe2712705 Merge 10.4 into 10.5
The functional changes of commit 5836191c8f
(MDEV-21168) are omitted due to MDEV-742 having addressed the issue.
2020-04-25 21:57:52 +03:00
Marko Mäkelä
af91266498 Merge 10.3 into 10.4
In main.index_merge_myisam we remove the test that was added in
commit a2d24def8c because
it duplicates the test case that was added in
commit 5af12e4635.
2020-04-16 12:12:26 +03:00
Eugene Kosov
6577a7a8f2 fix tests related to SQL comment length
tests are:
engines/funcs.jp_comment_column
engines/funcs.jp_comment_index
engines/funcs.jp_comment_table
2020-04-15 20:47:44 +03:00
Marko Mäkelä
37c14690fc Merge 10.4 into 10.5 2020-03-30 19:07:25 +03:00
Marko Mäkelä
e2f1f88fa6 Merge 10.3 into 10.4 2020-03-30 14:50:23 +03:00
Marko Mäkelä
1a9b6c4c7f Merge 10.2 into 10.3 2020-03-30 11:12:56 +03:00
Eugene Kosov
0b00c1a22f MDEV-22005 UBSAN: applying non-zero offset 2 to null pointer in my_charpos_mb()
Empty comment has a correct length.
2020-03-26 18:33:47 +03:00
Sergei Golubchik
7c58e97bf6 perfschema memory related instrumentation changes 2020-03-10 19:24:22 +01:00
Alexander Barkov
f1e13fdc8d MDEV-21581 Helper functions and methods for CHARSET_INFO 2020-01-28 12:29:23 +04:00
Alexander Barkov
d30dbaa20d A cleanup for MDEV-8844: Fixing compilation failure on Windows
Fixing lossy type conversions:
- from int64 to int
- from size_t to uint
2019-12-07 19:12:04 +04:00
Alexander Barkov
3c6065a270 MDEV-8844 Unreadable control characters printed as is in warnings 2019-12-06 18:51:05 +04:00
Aleksey Midenkov
0c05a2ed71 Merge 10.4 into 10.5 2019-11-25 17:24:09 +03:00
Marko Mäkelä
f9ceb0a67f MDEV-20190 Instant operation fails when add column and collation change on non-indexed column
We must relax too strict debug assertions. For latin1_swedish_ci,
mtype=DATA_CHAR or mtype=DATA_VARCHAR will be used instead of
mtype=DATA_MYSQL or mtype=DATA_VARMYSQL. Likewise, some changes of
dtype_get_charset_coll() do not affect the data type encoding,
but only any indexes that are defined on the column.

Charset::same_encoding(): Check whether two charset-collations have
the same character set encoding.

dict_col_t::same_encoding(): Check whether two character columns
have the same character set encoding.

dict_col_t::same_type(): Check whether two columns have a compatible
data type encoding.

dict_col_t::same_format(), dict_table_t::instant_column(): Do not
compare mtype or the charset-collation of prtype directly.
Rely on dict_col_t::same_type() instead.

dtype_get_charset_coll(): Narrow the return type to uint16_t.

This is a refined version of a fix that was developed by
Thirunarayanan Balathandayuthapani.
2019-11-25 15:26:22 +02:00
Alexander Barkov
9a833dc688 MDEV-20856 Bad values in metadata views for partitions on VARBINARY
The old code to print partition values was too complicated:
- it created new Items for character set conversion purposes.
- it mixed string conversion and partition error reporting
  in the same code blocks.

Simplifying the code as follows:

- Adding helper methods String::can_be_safely_convert_to() and
  String::append_introducer_and_hex().

- Adding DBUG_EXECUTE_IF("generate_partition_syntax_for_frm",  push_warning...)
  into generate_partition_syntax_for_frm(), to test the PARTITON
  clause written to FRM. Adding test partition_utf8-debug.test for this.

- Removing functions get_cs_converted_part_value_from_string() and
  get_cs_converted_string_value. Changing get_partition_column_description()
  to use Type_handler::partition_field_append_value() instead.
  This makes SHOW CREATE TABLE and SELECT FROM I_S.PARTITIONS
  use the same code path.

- Changing Type_handler::partition_field_append_value() not to
  call convert_charset_partition_constant(), to avoid creating a new Item
  for string conversion pursposes.
  Rewritting the code to use only String methods.

- Removing error reporting code (ER_PARTITION_FUNCTION_IS_NOT_ALLOWED)
  from Type_handler::partition_field_append_value().
  The error is correctly detected and reported on the caller level.
  So error reporting was redundant here.

Also:

- Moving methods Type_handler::partition_field_*() from sql_partition.cc
  to sql_type.cc. This fixes compilation problem with -DPLUGIN_PARTITION=NO,
  earlier introduced by the patch for MDEV-20831.
2019-10-18 10:15:17 +04:00
Marko Mäkelä
4081b7b27a Merge 10.4 into 10.5 2019-09-06 17:16:40 +03:00
Sergei Golubchik
244f0e6dd8 Merge branch '10.3' into 10.4 2019-09-06 11:53:10 +02:00
Alexander Barkov
7e08ac0b41 Merge 10.2 (up to commit ef00ac4c86) into 10.3 2019-09-04 10:19:58 +04:00
Alexander Barkov
dc719597ee MDEV-18156 Assertion 0' failed or btr_validate_index(index, 0, false)' in row_upd_sec_index_entry or error code 126: Index is corrupted upon DELETE with PAD_CHAR_TO_FULL_LENGTH
This change takes into account a column's GENERATED ALWAYS AS
expression dependcy on sql_mode's PAD_CHAR_TO_FULL_LENGTH and
NO_UNSIGNED_SUBTRACTION flags.

Indexed virtual columns as well as persistent generated columns are
now not allowed to have such dependencies to avoid inconsistent data
or index files on sql_mode changes.
So an error is now returned in cases like this:

  CREATE OR REPLACE TABLE t1
  (
    a CHAR(5),
    v VARCHAR(5) AS (a) PERSISTENT -- CHAR->VARCHAR or CHAR->TEXT = ERROR
  );

Functions RPAD() and RTRIM() can now remove dependency on
PAD_CHAR_TO_FULL_LENGTH. So this can be used instead:

  CREATE OR REPLACE TABLE t1
  (
    a CHAR(5),
    v VARCHAR(5) AS (RTRIM(a)) PERSISTENT
  );

Note, unlike CHAR->VARCHAR and CHAR->TEXT this still works,
not RPAD(a) is needed:

  CREATE OR REPLACE TABLE t1
  (
    a CHAR(5),
    v CHAR(5) AS (a) PERSISTENT -- CHAR->CHAR is OK
  );

More sql_mode flags may affect values of generated columns.
They will be addressed separately.

See comments in sql_mode.h for implementation details.
2019-09-03 05:34:53 +04:00
Alexander Barkov
1517087b54 MDEV-20042 Implement EXTRA2_FIELD_DATA_TYPE_INFO in FRM 2019-07-11 21:51:18 +04:00
Monty
1a41fc77dd Merge remote-tracking branch 'origin/10.4' into 10.5 2019-06-27 01:21:41 +03:00
Alexander Barkov
c62eaa7bdf MDEV-19843 Modify ST_FIELD_INFO to use Type_handler and LEX_CSTRING 2019-06-24 06:25:16 +04:00
Eugene Kosov
854c219a7f MDEV-17301 Change of COLLATE unnecessarily requires ALGORITHM=COPY
Patch is about two cases:
1) On some collate changes it's possible to rebuild only secondary indexes
2) For non-indexed columns collate can be changed INSTANTly

Implemented mostly in Field_{string,varstring,blob}::is_equal().
Make this method return how exactly collationa differs.
This information is later used by fill_alter_inplace_info() to pass
correct info to engine.
2019-06-22 14:09:12 +03:00
Varun Gupta
8b576616b4 MDEV-19776: Assertion `to_len >= 8' failed in convert_to_printable with optimizer trace enabled
Introduced the convert_to_printable_required_length to return the correct length(taking into
consideration of dots in the case of error messages).
2019-06-20 12:03:32 +05:30
Marko Mäkelä
984d7100cd Merge 10.4 into 10.5 2019-06-13 18:36:09 +03:00
Varun
a0cb7551a4 MDEV-18880: Optimizer trace prints date in hexadecimal
Introduced a print_key_value function to makes sure that the trace prints data in readable format
for readable characters and the rest of the characters are printed as hexadecimal.
2019-06-11 15:44:58 +05:30
Alexander Barkov
f42bda6d75 MDEV-19727 Add Type_handler::Key_part_spec_init_ft 2019-06-11 07:54:37 +04:00
Oleksandr Byelkin
c07325f932 Merge branch '10.3' into 10.4 2019-05-19 20:55:37 +02:00
Alexander Barkov
c59d6395a6 A joint patch for MDEV-19284 and MDEV-19285 (INSTANT ALTER)
This patch fixes:

- MDEV-19284 INSTANT ALTER with ucs2-to-utf16 conversion produces bad data
- MDEV-19285 INSTANT ALTER from ascii_general_ci to latin1_general_ci produces corrupt data

These regressions were introduced in 10.4.3 by:
- MDEV-15564 Avoid table rebuild in ALTER TABLE on collation or charset changes

Changes:

1. Cleanup: Adding a helper method
   Field_longstr::csinfo_change_allows_instant_alter(),
   to remove some duplicate code in field.cc.

2. Cleanup: removing Type_handler::Charsets_are_compatible() and static
   function charsets_are_compatible() and
   introducing new methods in the recently added class Charset instead:
   - encoding_allows_reinterpret_as()
   - encoding_and_order_allow_reinterpret_as()

3. Bug fix: Removing the code that allowed instant conversion for
   ascii-to->8bit and ucs2-to->utf16.
   This actually fixes MDEV-19284 and MDEV-19285.

4. Bug fix: Adding a helper method Charset::collation_specific_name().
   The old corresponding code in Type_handler::Charsets_are_compatible()
   was not safe against (badly named) user-defined collations whose
   character set name can be longer than collation name.
2019-05-16 16:20:25 +04:00
Marko Mäkelä
be85d3e61b Merge 10.2 into 10.3 2019-05-14 17:18:46 +03:00
Marko Mäkelä
26a14ee130 Merge 10.1 into 10.2 2019-05-13 17:54:04 +03:00
Vicențiu Ciorbaru
cb248f8806 Merge branch '5.5' into 10.1 2019-05-11 22:19:05 +03:00
Vicențiu Ciorbaru
5543b75550 Update FSF Address
* Update wrong zip-code
2019-05-11 21:29:06 +03:00
Marko Mäkelä
e6bdf77e4b Merge 10.3 into 10.4
In is_eits_usable(), we disable an assertion that fails due to
MDEV-19334.
2019-04-25 16:05:20 +03:00
Marko Mäkelä
acf6f92aa9 Merge 10.2 into 10.3 2019-04-25 09:05:52 +03:00
Marko Mäkelä
bc145193c1 Merge 10.1 into 10.2 2019-04-25 09:04:09 +03:00
Alexander Barkov
9dcfd6be94 MDEV-9465 The constructor StringBuffer(const char *str, size_t length, const CHARSET_INFO *cs) looks suspicious
Removing the suspicious constructor, it's not used in the code anyway.
2019-04-23 09:11:42 +04:00
Marko Mäkelä
514b305dfb Merge 10.3 into 10.4
The MDEV-17262 commit 26432e49d3
was skipped. In Galera 4, the implementation would seem to require
changes to the streaming replication.

In the tests archive.rnd_pos main.profiling, disable_ps_protocol
for SHOW STATUS and SHOW PROFILE commands until MDEV-18974
has been fixed.
2019-03-20 10:41:32 +02:00
Sergei Golubchik
b64fde8f38 Merge branch '10.2' into 10.3 2019-03-17 13:06:41 +01:00
Sergei Golubchik
f1134d5676 post-merge: gcc 8 warnings
note: Inherit String from Sql_alloc,
to get operators new and new[] in sync

in rocksdb gcc was complaining that non-lvalue was cast to const.
2019-03-15 21:00:50 +01:00
Alexander Barkov
bb9b4182e4 MDEV-17906 Class Binary_string 2018-12-05 15:28:08 +04:00