This is a joint patch for:
MDEV-12852 Out-of-range errors when CAST(1-2 AS UNSIGNED
MDEV-12853 Out-of-range errors when CAST('-1' AS UNSIGNED
MDEV-12869 Wrong metadata for integer additive and multiplicative operators
1. Fixing all Item_func_numhybrid descendants to set the precise
data type handler (type_handler_long or type_handler_longlong)
at fix_fields() time. This fixes MDEV-12869.
2. Fixing Item_func_unsigned_typecast to set the precise data type handler
at fix_fields() time. This fixes MDEV-12852 and MDEV-12853.
This is done by:
- fixing Type_handler::Item_func_unsigned_fix_length_and_dec()
and Type_handler_string_result::Item_func_unsigned_fix_length_and_dec()
to properly detect situations when a negative epxression is converted
to UNSIGNED. In this case, length of the result is now always set to
MAX_BIGINT_WIDTH without trying to use args[0]->max_length, as very
short arguments can produce very long result in such conversion:
CAST(-1 AS UNSIGNED) -> 18446744073709551614
- adding a new virtual method "longlong Item::val_int_max() const",
to preserve the old behavior for expressions like this:
CAST(1 AS UNSIGNED)
to stay under the INT data type (instead of BIGINT) for small
positive integer literals. Using Item::unsigned_flag would not help,
because Item_int does not set unsigned_flag to "true" for positive
numbers.
3. Adding helper methods:
* Item::type_handler_long_or_longlong()
* Type_handler::type_handler_long_or_longlong()
and reusing them in a few places, to reduce code duplication.
4. Making reorganation in create_tmp_field() and
create_field_for_create_select() for Item_hybrid_func and descendants,
to reduce duplicate code. They all now have a similar behavior in
respect of creating fields. Only Item_func_user_var descendants have
a different behavior. So moving the default behvior to Item_hybrid_func,
and overriding behavior on Item_func_user_var level.
Do not silence uncertain cases, or fix any bugs.
The only functional change should be that ha_federated::extra()
is not calling DBUG_PRINT to report an unhandled case for
HA_EXTRA_PREPARE_FOR_DROP.
Do not silence uncertain cases, or fix any bugs.
The only functional change should be that ha_federated::extra()
is not calling DBUG_PRINT to report an unhandled case for
HA_EXTRA_PREPARE_FOR_DROP.
Introducing a new class Type_holder (used internally in sql_union.cc),
to reuse exactly the same data type attribute aggregation Type_handler API
for hybrid functions and UNION.
This fixes a number of bugs in UNION:
- MDEV-9495 Wrong field type for a UNION of a signed and an unsigned INT expression
- MDEV-9497 UNION and COALESCE produce different field types for DECIMAL+INT
- MDEV-12594 UNION between fixed length double columns does not always preserve scale
- MDEV-12595 UNION converts INT to BIGINT
- MDEV-12599 UNION is not symmetric when mixing INT and CHAR
Details:
- sql_union.cc: Reusing attribute aggregation for UNION.
Adding new methods:
* st_select_lex_unit::join_union_type_handlers()
* st_select_lex_unit::join_union_type_attributes()
* st_select_lex_unit::join_union_item_types()
Removing the old join_types()-based code.
- Changing Type_handler::Item_hybrid_func_fix_attributes()
to accept "name", Type_handler_hybrid_field_type, Type_all_attributes
as three separate parameters instead of a single Item_hybrid_func parameter,
to make it possible to pass both Item_hybrid_func and Type_holder.
- Moving the former special GEOMETRY and ENUM/SET attribute aggregation code
from Item_type_holder::join_types() to
* Type_handler_typelib::Item_hybrid_func_fix_attributes().
* Type_handler_geometry::Item_hybrid_func_fix_attrubutes().
This makes GEOMETRY/ENUM/SET symmetric with all other data types
(from the UNION point of view).
Removing Item_type_holder::join_types() and Item_type_holder::get_full_info().
- Adding new methods into Type_all_attributes:
* Type_all_attributes::set_geometry_type() and
Item_hybrid_func::set_geometry_type().
* Adding Type_all_attributes::get_typelib().
* Adding Type_all_attributes::set_typelib().
- Adding Type_handler_typelib as a common parent for
Type_handler_enum and Type_handler_set, to avoid code duplication: they have
already had two common methods, and we're adding one more shared method.
- Adding Type_all_attributes::set_maybe_null(), as some type handlers
may want to set maybe_null (e.g. Type_handler_geometry) during data type
attribute aggregation.
- Changing Type_geometry_attributes() to accept Type_handler
and Type_all_attributes as two separate parameters, instead
of a single Item parameter, to make it possible to pass Type_holder.
- Adding Item_args::add_argument().
- Moving Item_args::alloc_arguments() from "protected" to "public".
- Moving Item_type_holder::Item_type_holder() from item.cc to item.h, as
now it's very simple.
Btw, this constructor should probably be eventually removed.
It's now used only in sql_show.cc, which could be modified to use
Item_return_decimal (for symmetry with Item_return_xxx created for all
other data types). Or, another option: remove all Item_return_xxx and
use Item_type_holder for all data types instead.
- storage/tokudb/mysql-test/tokudb/r/type_float.result
Recording new results (MDEV-12594).
- mysql-test/r/cte_recursive.result
Recording new results (MDEV-9497)
- mysql-test/r/subselect*.result
Recording new results (MDEV-12595)
- mysql-test/r/metadata.result
Recording new results (MDEV-9495)
- mysql-test/r/temp_table.result
Recording new results (MDEV-12594)
- mysql-test/r/type_float.result
Recording new results (MDEV-12594)
- SETVAL(sequence_name, next_value, is_used, round)
- ALTER SEQUENCE, including RESTART WITH
Other things:
- Added handler::extra() option HA_EXTRA_PREPARE_FOR_ALTER_TABLE to signal
ha_sequence() that it should allow write_row statments.
- ALTER ONLINE TABLE now works with SEQUENCE:s
- Adding Type_handler::make_table_field() and moving pieces of the code
from Item::tmp_table_field_from_field_type() to virtual implementations
for various type handlers.
- Adding a new Type_all_attributes, to access to Item's extended
attributes, such as decimal_precision() and geometry_type().
- Adding a new class Record_addr, to pass record related information
to Type_handler methods (ptr, null_ptr and null_bit) as a single structure.
Note, later it will possibly be extended for BIT-alike field purposes,
by adding new members (bit_ptr_arg, bit_ofs_arg).
- Moving the code from Field_new_decimal::create_from_item()
to Type_handler_newdecimal::make_table_field().
- Removing Field_new_decimal() and Field_geom() helper constructor
variants that were used for temporary field creation.
- Adding Item_field::type_handler(), Field::type_handler() and
Field_blob::type_handler() to return correct type handlers for
blob variants, according to Field_blob::packlength.
- Adding Type_handler_blob_common, as a common parent for
Type_handler_tiny_blob, Type_handler_blob, Type_handler_medium_blob
and Type_handler_long_blob.
- Implementing Type_handler_blob_common::Item_hybrid_func_fix_attributes().
It's needed for cases when TEXT variants of different character sets are mixed
in LEAST, GREATEST, CASE and its abreviations (IF, IFNULL, COALESCE), e.g.:
CREATE TABLE t1 (
a TINYTEXT CHARACTER SET latin1,
b TINYTEXT CHARACTER SET utf8
);
CREATE TABLE t2 AS SELECT COALESCE(a,b) FROM t1;
Type handler aggregation returns TINYTEXT as a common data type
for the two columns. But as conversion from latin1 to utf8
happens for "a", the maximum possible length of "a" grows from 255 to 255*3.
Type_handler_blob_common::Item_hybrid_func_fix_attributes() makes sure
to update the blob type handler according to max_length.
- Adding Type_handler::blob_type_handler(uint max_octet_length).
- Adding a few m_type_aggregator_for_result.add() pairs, because
now Item_xxx::type_handler() can return pointers to type_handler_tiny_blob,
type_handler_blob, type_handler_medium_blob, type_handler_long_blob.
Before the patch only type_handler_blob was possible result of type_handler().
- Making type_handler_tiny_blob, type_handler_blob, type_handler_medium_blob,
type_handler_long_blob public.
- Removing the condition in Item_sum_avg::create_tmp_field()
checking Item_sum_avg::result_type() against DECIMAL_RESULT.
Now both REAL_RESULT and DECIMAL_RESULT are symmetrically handled
by tmp_table_field_from_field_type().
- Removing Item_geometry_func::create_field_for_create_select(),
as the inherited version perfectly works.
- Fixing Item_func_as_wkb::field_type() to return MYSQL_TYPE_LONG_BLOB
rather than MYSQL_TYPE_BLOB. It's needed to make sure that
tmp_table_field_from_field_type() creates a LONGBLOB field for AsWKB().
- Fixing Item_func_as_wkt::fix_length_and_dec() to set max_length to
UINT32_MAX rather than MAX_BLOB_WIDTH, to make sure that
tmp_table_field_from_field_type() creates a LONGTEXT field for AsWKT().
- Removing Item_func_set_user_var::create_field_for_create_select(),
as the inherited version works fine.
- Adding Item_func_get_user_var::create_field_for_create_select() to
make sure that "CREATE TABLE t1 AS SELECT @string_user variable"
always creates a field of LONGTEXT/LONGBLOB type.
- Item_func_ifnull::create_field_for_create_select()
behavior has changed. Before the patch it passed set_blob_packflag=false,
which meant to create LONGBLOB for all blob variants.
Now it takes into account max_length, which gives better column
data types for:
CREATE TABLE t2 AS SELECT IFNULL(blob_column1, blob_column2) FROM t1;
- Fixing Item_func_nullif::fix_length_and_dec() to use
set_handler(args[2]->type_handler()) instead of
set_handler_by_field_type(args[2]->field_type()).
This is needed to distinguish between BLOB variants.
- Implementing Item_blob::type_handler(), to make sure to create
proper BLOB field variant, according to max_length, for queries like:
CREATE TABLE t1 AS
SELECT some_blob_field FROM INFORMATION_SCHEMA.SOME_TABLE;
- Fixing Item_field::real_type_handler() to make sure that
the code aggregating fields for UNION gets a proper BLOB
variant type handler from fields.
- Adding a special code into Item_type_holder::make_field_by_type(),
to make sure that after aggregating field types it also properly
takes into account max_length when mixing TEXT variants of different
character sets and chooses a proper TEXT variant:
CREATE TABLE t1 (
a TINYTEXT CHARACTER SET latin1,
b TINYTEXT CHARACTER SET utf8
);
CREATE TABLE t2 AS SELECT a FROM t1 UNION SELECT b FROM t1;
- Adding tests, for better coverage of IFNULL, NULLIF, UNION.
- The fact that tmp_table_field_from_field_type() now takes
into account BLOB variants (instead of always creating LONGBLOB),
tests results for WEIGHT_STRING() and NULLIF() and UNION
have become more precise.
Benefits of this patch:
- Removed a lot of calls to strlen(), especially for field_string
- Strings generated by parser are now const strings, less chance of
accidently changing a string
- Removed a lot of calls with LEX_STRING as parameter (changed to pointer)
- More uniform code
- Item::name_length was not kept up to date. Now fixed
- Several bugs found and fixed (Access to null pointers,
access of freed memory, wrong arguments to printf like functions)
- Removed a lot of casts from (const char*) to (char*)
Changes:
- This caused some ABI changes
- lex_string_set now uses LEX_CSTRING
- Some fucntions are now taking const char* instead of char*
- Create_field::change and after changed to LEX_CSTRING
- handler::connect_string, comment and engine_name() changed to LEX_CSTRING
- Checked printf() related calls to find bugs. Found and fixed several
errors in old code.
- A lot of changes from LEX_STRING to LEX_CSTRING, especially related to
parsing and events.
- Some changes from LEX_STRING and LEX_STRING & to LEX_CSTRING*
- Some changes for char* to const char*
- Added printf argument checking for my_snprintf()
- Introduced null_clex_str, star_clex_string, temp_lex_str to simplify
code
- Added item_empty_name and item_used_name to be able to distingush between
items that was given an empty name and items that was not given a name
This is used in sql_yacc.yy to know when to give an item a name.
- select table_name."*' is not anymore same as table_name.*
- removed not used function Item::rename()
- Added comparision of item->name_length before some calls to
my_strcasecmp() to speed up comparison
- Moved Item_sp_variable::make_field() from item.h to item.cc
- Some minimal code changes to avoid copying to const char *
- Fixed wrong error message in wsrep_mysql_parse()
- Fixed wrong code in find_field_in_natural_join() where real_item() was
set when it shouldn't
- ER_ERROR_ON_RENAME was used with extra arguments.
- Removed some (wrong) ER_OUTOFMEMORY, as alloc_root will already
give the error.
TODO:
- Check possible unsafe casts in plugin/auth_examples/qa_auth_interface.c
- Change code to not modify LEX_CSTRING for database name
(as part of lower_case_table_names)
This patch does the following:
1. Adds a new method Type_handler_hybrid_field_type::aggregate_for_min_max()
- For non-traditional data types it uses
type_handler_data->m_type_aggregator_for_result.find_handler()
This allows pluggable data types to define in the future their
own behavior of the result data type detection for LEAST/GREATEST.
Also, this disallows expressions of the GEOMETRY data type
(and its variants such as POINT) to be mixed in with
numeric and temporal data types in LEAST/GREATEST.
- For traditional data types it reproduces the old behavior of
the result data type detection (but not attributes, see below).
2. Adds a new virtual method Type_handler::Item_func_min_max_fix_attributes()
and reuses as much as possible the code that calculates data type attributes
for CASE-alike functions (e.g. CASE..THEN, COALESCE, IF).
As the old code responsible for attributes calculation in the old
implementation of Item_func_min_max::fix_length_and_dec()
was not fully correct, this automatically fixes the following bugs:
- MDEV-12497 Wrong data type for LEAST(latin1_expr, utf8_expr)
The old fix_length_and_dec() calculated max_length before
character set aggregation. Now max_length is calculated after, in
Item_func::count_string_length() called from
Item_func::aggregate_attributes_string() called from
Type_handler_string_result::Item_hybrid_func_fix_attributes() called from
Type_handler::Item_func_min_max_fix_attributes() called from
Item_func_min_max::fix_length_and_dec().
- MDEV-12504 Wrong data type for LEAST(date_expr,time_expr)
The old fix_length_and_dec() simply used the maximum of max_length
among all arguments to set its own max_length and did not take
into account that a mixture of DATE and TIME becomes DATETIME.
Now this is correctly handled by:
Type_handler_datetime_common::Item_hybrid_func_fix_attributes() called from
Type_handler::Item_func_min_max_fix_attributes() called from
Item_func_min_max::fix_length_and_dec().
3. Removes the old implementation of Item_func_min_max::fix_length_and_dec()
and replaces it to calls of the new methods.
4. Cleanup: moves the code related to unsigned_flag processing
from Type_handler_hybrid_field_type::aggregate_for_result()
to Type_handler_int_result::Item_hybrid_func_fix_attributes().
This is done:
- to avoid code duplication in
Type_handler_hybrid_field_type::aggregate_for_min_max()
- to get rid of one more call for field_type(), which is unfriendly
to the conceipt of pluggable data types.
Working features:
CREATE OR REPLACE [TEMPORARY] SEQUENCE [IF NOT EXISTS] name
[ INCREMENT [ BY | = ] increment ]
[ MINVALUE [=] minvalue | NO MINVALUE ]
[ MAXVALUE [=] maxvalue | NO MAXVALUE ]
[ START [ WITH | = ] start ] [ CACHE [=] cache ] [ [ NO ] CYCLE ]
ENGINE=xxx COMMENT=".."
SELECT NEXT VALUE FOR sequence_name;
SELECT NEXTVAL(sequence_name);
SELECT PREVIOUS VALUE FOR sequence_name;
SELECT LASTVAL(sequence_name);
SHOW CREATE SEQUENCE sequence_name;
SHOW CREATE TABLE sequence_name;
CREATE TABLE sequence-structure ... SEQUENCE=1
ALTER TABLE sequence RENAME TO sequence2;
RENAME TABLE sequence TO sequence2;
DROP [TEMPORARY] SEQUENCE [IF EXISTS] sequence_names
Missing features
- SETVAL(value,sequence_name), to be used with replication.
- Check replication, including checking that sequence tables are marked
not transactional.
- Check that a commit happens for NEXT VALUE that changes table data (may
already work)
- ALTER SEQUENCE. ANSI SQL version of setval.
- Share identical sequence entries to not add things twice to table list.
- testing insert/delete/update/truncate/load data
- Run and fix Alibaba sequence tests (part of mysql-test/suite/sql_sequence)
- Write documentation for NEXT VALUE / PREVIOUS_VALUE
- NEXTVAL in DEFAULT
- Ensure that NEXTVAL in DEFAULT uses database from base table
- Two NEXTVAL for same row should give same answer.
- Oracle syntax sequence_table.nextval, without any FOR or FROM.
- Sequence tables are treated as 'not read constant tables' by SELECT; Would
be better if we would have a separate list for sequence tables so that
select doesn't know about them, except if refereed to with FROM.
Other things done:
- Improved output for safemalloc backtrack
- frm_type_enum changed to Table_type
- Removed lex->is_view and replaced with lex->table_type. This allows
use to more easy check if item is view, sequence or table.
- Added table flag HA_CAN_TABLES_WITHOUT_ROLLBACK, needed for handlers
that want's to support sequences
- Added handler calls:
- engine_name(), to simplify getting engine name for partition and sequences
- update_first_row(), to be able to do efficient sequence implementations.
- Made binlog_log_row() global to be able to call it from ha_sequence.cc
- Added handler variable: row_already_logged, to be able to flag that the
changed row is already logging to replication log.
- Added CF_DB_CHANGE and CF_SCHEMA_CHANGE flags to simplify
deny_updates_if_read_only_option()
- Added sp_add_cfetch() to avoid new conflicts in sql_yacc.yy
- Moved code for add_table_options() out from sql_show.cc::show_create_table()
- Added String::append_longlong() and used it in sql_show.cc to simplify code.
- Added extra option to dd_frm_type() and ha_table_exists to indicate if
the table is a sequence. Needed by DROP SQUENCE to not drop a table.
