This is a joint patch fixing the following problems:
MDEV-12875 Wrong VIEW column data type for COALESCE(int_column)
MDEV-12886 Different default for INT and BIGINT column in a VIEW for a SELECT with ROLLUP
MDEV-12916 Wrong column data type for an INT field of a cursor-anchored ROW variable
All above problem happened because the global function ::create_tmp_field()
called the top-level Item::create_tmp_field(), which made some tranformation
for INT-result data types. For example, INT(11) became BIGINT(11), because 11
is a corner case and it's not known if it fits or does not fit into INT range,
so Item::create_tmp_field() converted it to BIGINT(11) for safety.
The main idea of this patch is to avoid such tranformations.
1. Fixing Item::create_tmp_field() not to have a special case for INT_RESULT.
Item::create_tmp_field() is changed not to have a special case
for INT_RESULT (which earlier made a decision based on Item's max_length).
It now calls tmp_table_field_from_field_type() for INT_RESULT,
therefore preserves the original data type (e.g. INT, YEAR) without
conversion to BIGINT.
This change is valid, because a number of recent fixes
(e.g. in Item_func_int, Item_hybrid_func, Item_int, Item_splocal)
guarantee that item->type_handler() now properly returns
type_handler_long vs type_handler_longlong. So no adjustment by length
is needed any more for Items returning INT_RESULT.
After this change, Item::create_tmp_field() calls
tmp_table_field_from_field_type() for all XXX_RESULT, except REAL_RESULT.
2. Fixing Item::create_tmp_field() not to have a special case for REAL_RESULT.
Note, the reason for a special case for REAL_RESULT is to have a special
constructor for Field_double(), forcing Field_real::not_fixed to be set
to true.
Taking into account that only Item_sum descendants actually need a special
constructor call Field_double(not_fixed=true), not too loose precision
when mixing individual rows to the aggregate result:
- renaming Item::create_tmp_field() to Item_sum::create_tmp_field().
- changing Item::create_tmp_field() just to call
tmp_table_field_from_field_type() for all XXX_RESULT types.
A special case for REAL_RESULT in Item::create_tmp_field() is now gone.
Item::create_tmp_field() is now symmetric for all XXX_RESULT types,
and now just calls tmp_table_field_from_field_type().
3. Fixing Item_func::create_field_for_create_select() not to have
a special case for STRING_RESULT.
After changes #1 and #2, the code in
Item_func::create_field_for_create_select(), testing result_type(),
becomes useless, because: now Item::create_tmp_field() and
tmp_table_field_from_field_type() do exactly the same thing for all
XXX_RESULT types for Item_func descendants:
a. It calls tmp_table_field_from_field_type for STRING_RESULT directly.
b. For other XXX_RESULT, it goes through Item::create_tmp_field(),
which calls the global function ::create_tmp_field(),
which calls item->create_tmp_field() for FUNC_ITEM,
which calls tmp_table_field_from_field_type() again.
So removing the virtual implementation of
Item_func::create_field_for_create_select().
The inherited Item::create_field_for_create_select() now perfectly
does the job, as it also calls tmp_table_field_from_field_type()
for FUNC_ITEM, independently from XXX_RESULT type.
4. Taking into account #1 and #2, as well as some recent changes,
removing virtual implementations:
- Item_hybrid_func::create_tmp_field()
- Item_hybrid_func::create_field_for_create_select()
- Item_int_func::create_tmp_field()
- Item_int_func::create_field_for_create_select()
- Item_temporal_func::create_field_for_create_select()
The derived versions from Item now perfectly work.
5. Moving a piece of code from create_tmp_field_from_item()
to a new function create_tmp_field_from_item_finalize(),
to reuse it in two places (see #6).
6. Changing the code responsible for BIT->INT/BIGIN tranformation
(which is called for the cases when the created table, e.g. HEAP,
does not fully support BIT) not to call create_tmp_field_from_item(),
because the latter now calls tmp_table_field_from_field_type() instead
of create_tmp_field() and thefore cannot do BIT transformation.
So rewriting this code using a sequence of these calls:
- item->type_handler_long_or_longlong()
- handler->make_and_init_table_field()
- create_tmp_field_from_item_finalize()
7. Miscelaneous changes:
- Moving type_handler_long_or_longlong() from "protected" to "public",
as it's now needed in the global function create_tmp_field().
8. The above changes fixed MDEV-12875, MDEV-12886, MDEV-12916.
So adding tests for these bugs.
- Test with LOCK TABLES
- Test mysqldump
- Don't update rows for sequence tables if values doesn't change. This is
needed as InnoDB gives an error for updates where values doesn't change.
Because SEQUENCE objects or NO_ROLLBACK tables do not support locking
or MVCC or transactions, avoid starting a transaction.
row_upd_step(): Do not start a transaction. Let the caller do that.
que_thr_step(): Call trx_start_if_not_started_xa() for QUE_NODE_UPDATE.
(The InnoDB SQL parser is not used for accessing NO_ROLLBACK tables.)
row_ins_step(): Correct a too strict assertion and comment about
concurrency. Multiple concurrent readers are allowed.
row_update_for_mysql_using_upd_graph(): Do not start the transaction
for NO_ROLLBACK tables.
row_search_mvcc(): For NO_ROLLBACK tables, skip locking even inside
LOCK TABLES. Only call trx_start_if_not_started() at the start
of a statement, not for each individual request.
The patch broke expressions like CAST(1.0e+300 AS SIGNED INT)
in binary protocol, e.g.:
mtr --ps cast
Short real numbers like 1.0e+300 can return huge values,
so using args[0]->max_length is not reliable to choose properly the result
type for Item_func_signed and Item_func_unsigned (between INT and BIGINT).
Setting Item_[un]signed_typecast::max_length to MAX_BIGINT_WIDTH
when doing CAST from FLOAT/DOUBLE, to force type_handler() return
&type_handler_longlong rather than &type_handler_long.
- Old sequence code forced row based replication for any statements that
refered to a sequence table. What is new is that row based replication
is now sequence aware:
- NEXT VALUE is now generating a short row based event with only
next_value and round being replicated.
- Short row based events are now on the slave updated as trough
SET_VALUE(sequence_name)
- Full row based events are on the slave updated with a full insert,
which is practically same as ALTER SEQUENCE.
- INSERT on a SEQUENCE table does now a EXCLUSIVE LOCK to ensure that
it is logged in binary log before any following NEXT VALUE calls.
- Enable all sequence tests and fixed found bugs
- ALTER SEQUENCE doesn't anymore allow changes that makes the next_value
outside of allowed range
- SEQUENCE changes are done with TL_WRITE_ALLOW_WRITE. Because of this
one can generate a statement for MyISAM with both
TL_WRITE_CONCURRENT_INSERT and TL_WRITE_ALLOW_WRITE. To fix a warning
I had to add an extra test in thr_lock.c for this.
- Removed UPDATE of SEQUENCE (no need to support this as we
have ALTER SEQUENCE, which takes the EXCLUSIVE lock properly.
- Removed DBUG_ASSERT() in MDL_context::upgrade_shared_lock. This was
removed upstream in MySQL 5.6 in 72f823de453.
- Simplified test in decided_logging_format() by using sql_command_flags()
- Fix that we log DROP SEQUENCE correctly.
- Fixed that Aria works with SEQUENCE
This is a joint patch for:
MDEV-12852 Out-of-range errors when CAST(1-2 AS UNSIGNED
MDEV-12853 Out-of-range errors when CAST('-1' AS UNSIGNED
MDEV-12869 Wrong metadata for integer additive and multiplicative operators
1. Fixing all Item_func_numhybrid descendants to set the precise
data type handler (type_handler_long or type_handler_longlong)
at fix_fields() time. This fixes MDEV-12869.
2. Fixing Item_func_unsigned_typecast to set the precise data type handler
at fix_fields() time. This fixes MDEV-12852 and MDEV-12853.
This is done by:
- fixing Type_handler::Item_func_unsigned_fix_length_and_dec()
and Type_handler_string_result::Item_func_unsigned_fix_length_and_dec()
to properly detect situations when a negative epxression is converted
to UNSIGNED. In this case, length of the result is now always set to
MAX_BIGINT_WIDTH without trying to use args[0]->max_length, as very
short arguments can produce very long result in such conversion:
CAST(-1 AS UNSIGNED) -> 18446744073709551614
- adding a new virtual method "longlong Item::val_int_max() const",
to preserve the old behavior for expressions like this:
CAST(1 AS UNSIGNED)
to stay under the INT data type (instead of BIGINT) for small
positive integer literals. Using Item::unsigned_flag would not help,
because Item_int does not set unsigned_flag to "true" for positive
numbers.
3. Adding helper methods:
* Item::type_handler_long_or_longlong()
* Type_handler::type_handler_long_or_longlong()
and reusing them in a few places, to reduce code duplication.
4. Making reorganation in create_tmp_field() and
create_field_for_create_select() for Item_hybrid_func and descendants,
to reduce duplicate code. They all now have a similar behavior in
respect of creating fields. Only Item_func_user_var descendants have
a different behavior. So moving the default behvior to Item_hybrid_func,
and overriding behavior on Item_func_user_var level.
MDEV-12858 Out-of-range error for CREATE..SELECT unsigned_int_column+1
MDEV-12859 Out-of-range error for CREATE..SELECT @a:=EXTRACT(MINUTE_MICROSECOND FROM..)
MDEV-12862 Data type of @a:=1e0 depends on the session character set
1. Moving a part of Item::create_tmp_field() into a new helper method
Item::create_tmp_field_int() and reusing it in Item::create_tmp_field()
and Item_func_signed::create_tmp_field().
Fixing the code in Item::create_tmp_field_int() to call
Type_handler::make_table_field() instead of doing "new Field_long[long]"
directly. This change revealed a problem reported in MDEV-12862.
2. Changing the "long vs longlong" cut-off length for
- Item_func::create_tmp_field()
- Item_sum::create_tmp_field()
- Item_func_get_user_var::create_tmp_field()
from MY_INT32_NUM_DECIMAL_DIGITS to (MY_INT32_NUM_DECIMAL_DIGITS - 2).
This fixes MDEV-12858.
After this change, the "convert_int_length" parameter to
Item::create_tmp_field() is not needed any more, because
(MY_INT32_NUM_DECIMAL_DIGITS - 2) is always passed.
So removing the "convert_int_length" parameter.
3. Fixing Item::create_tmp_field() to pass max_char_length() instead
of max_length to the constructor of Field_double().
This fixes MDEV-12862.
4. Additionally, fixing
- Type_handler_{tiny|short|int24|long|longlong}::make_table_field()
- Type_handler_{float|double}::make_table_field()
to pass max_char_length() instead of max_length to Field contructors.
This is needed by the change (1).
5. Adding new tests, and recording new correct results in the old tests in:
- mysql-test/r/type_ranges.result
- storage/tokudb/mysql-test/tokudb/r/type_ranges.result
1. Adding the forgotten "SET sql_mode=STRICT_ALL_TABLES" into the test.
2. STRICT_ALL_TABLES revealed that CAST(0xFFFFFFFF AS SIGNED),
e.g. with a hex number with 8 hex digits, did not work well.
Fixing Item_func_unsigned::create_tmp_field() and
Item_func_unsigned::create_field_for_create_select() to handle
this corner case.
- Adding new virtual methods in Type_handler:
* Column_definition_prepare_stage1()
* Column_definition_prepare_stage2()
* calc_pack_length()
- Using new methods to remove type specific code in:
* Global function calc_pack_length()
* Column_definition::prepare_create_field()
* The loop body mysql_prepare_create_table()
* Column_definition::sp_prepare_create_field()
Introducing a new class Type_holder (used internally in sql_union.cc),
to reuse exactly the same data type attribute aggregation Type_handler API
for hybrid functions and UNION.
This fixes a number of bugs in UNION:
- MDEV-9495 Wrong field type for a UNION of a signed and an unsigned INT expression
- MDEV-9497 UNION and COALESCE produce different field types for DECIMAL+INT
- MDEV-12594 UNION between fixed length double columns does not always preserve scale
- MDEV-12595 UNION converts INT to BIGINT
- MDEV-12599 UNION is not symmetric when mixing INT and CHAR
Details:
- sql_union.cc: Reusing attribute aggregation for UNION.
Adding new methods:
* st_select_lex_unit::join_union_type_handlers()
* st_select_lex_unit::join_union_type_attributes()
* st_select_lex_unit::join_union_item_types()
Removing the old join_types()-based code.
- Changing Type_handler::Item_hybrid_func_fix_attributes()
to accept "name", Type_handler_hybrid_field_type, Type_all_attributes
as three separate parameters instead of a single Item_hybrid_func parameter,
to make it possible to pass both Item_hybrid_func and Type_holder.
- Moving the former special GEOMETRY and ENUM/SET attribute aggregation code
from Item_type_holder::join_types() to
* Type_handler_typelib::Item_hybrid_func_fix_attributes().
* Type_handler_geometry::Item_hybrid_func_fix_attrubutes().
This makes GEOMETRY/ENUM/SET symmetric with all other data types
(from the UNION point of view).
Removing Item_type_holder::join_types() and Item_type_holder::get_full_info().
- Adding new methods into Type_all_attributes:
* Type_all_attributes::set_geometry_type() and
Item_hybrid_func::set_geometry_type().
* Adding Type_all_attributes::get_typelib().
* Adding Type_all_attributes::set_typelib().
- Adding Type_handler_typelib as a common parent for
Type_handler_enum and Type_handler_set, to avoid code duplication: they have
already had two common methods, and we're adding one more shared method.
- Adding Type_all_attributes::set_maybe_null(), as some type handlers
may want to set maybe_null (e.g. Type_handler_geometry) during data type
attribute aggregation.
- Changing Type_geometry_attributes() to accept Type_handler
and Type_all_attributes as two separate parameters, instead
of a single Item parameter, to make it possible to pass Type_holder.
- Adding Item_args::add_argument().
- Moving Item_args::alloc_arguments() from "protected" to "public".
- Moving Item_type_holder::Item_type_holder() from item.cc to item.h, as
now it's very simple.
Btw, this constructor should probably be eventually removed.
It's now used only in sql_show.cc, which could be modified to use
Item_return_decimal (for symmetry with Item_return_xxx created for all
other data types). Or, another option: remove all Item_return_xxx and
use Item_type_holder for all data types instead.
- storage/tokudb/mysql-test/tokudb/r/type_float.result
Recording new results (MDEV-12594).
- mysql-test/r/cte_recursive.result
Recording new results (MDEV-9497)
- mysql-test/r/subselect*.result
Recording new results (MDEV-12595)
- mysql-test/r/metadata.result
Recording new results (MDEV-9495)
- mysql-test/r/temp_table.result
Recording new results (MDEV-12594)
- mysql-test/r/type_float.result
Recording new results (MDEV-12594)
"mtr --ps func_str" failed:
pad_str was erroneously appended with a new space on every PS execution.
Adding pad_str.length(0) into fix_length_and_dec() to avoid this.
- SETVAL(sequence_name, next_value, is_used, round)
- ALTER SEQUENCE, including RESTART WITH
Other things:
- Added handler::extra() option HA_EXTRA_PREPARE_FOR_ALTER_TABLE to signal
ha_sequence() that it should allow write_row statments.
- ALTER ONLINE TABLE now works with SEQUENCE:s
Other things
- Ensure that ut_d() is set to EXPR if ut_ad() is DEBUG_ASSERT()
If not, we will get a crash in purge_sys_t::~purge_sys_t() as
this ut_ad() code expect's that the ut_d() codes has been executed
