Commit graph

1464 commits

Author SHA1 Message Date
Sachin
d00f19e832 MDEV-371 Unique Index for long columns
This patch implements engine independent unique hash index.

Usage:- Unique HASH index can be created automatically for blob/varchar/test column whose key
 length > handler->max_key_length()
or it can be explicitly specified.

  Automatic Creation:-
   Create TABLE t1 (a blob unique);
  Explicit Creation:-
   Create TABLE t1 (a int , unique(a) using HASH);

Internal KEY_PART Representations:-
 Long unique key_info will have 2 representations.
 (lets understand this with an example create table t1(a blob, b blob , unique(a, b)); )

 1. User Given Representation:- key_info->key_part array will be similar to what user has defined.
 So in case of example it will have 2 key_parts (a, b)

 2. Storage Engine Representation:- In this case there will be only one key_part and it will point to
 HASH_FIELD. This key_part will be always after user defined key_parts.

 So:- User Given Representation          [a] [b] [hash_key_part]
                  key_info->key_part ----^
  Storage Engine Representation          [a] [b] [hash_key_part]
                  key_info->key_part ------------^

 Table->s->key_info will have User Given Representation, While table->key_info will have Storage Engine
 Representation.Representation can be changed into each other by calling re/setup_keyinfo_hash function.

Working:-

1. So when user specifies HASH_INDEX or key_length is > handler->max_key_length(), In mysql_prepare_create_table
One extra vfield is added (for each long unique key). And key_info->algorithm is set to HA_KEY_ALG_LONG_HASH.

2. In init_from_binary_frm_image values for hash_keypart is set (like fieldnr , field and flags)

3. In parse_vcol_defs, HASH_FIELD->vcol_info is created. Item_func_hash is used with list of Item_fields,
   When Explicit length is given by user then Item_left is used to concatenate Item_field values.

4. In ha_write_row/ha_update_row check_duplicate_long_entry_key is called which will create the hash key from
table->record[0] and then call ha_index_read_map , if we found duplicated hash , we will compare the result
field by field.
2019-02-22 00:35:40 +01:00
Oleksandr Byelkin
93ac7ae70f Merge branch '10.3' into 10.4 2019-02-21 14:40:52 +01:00
Galina Shalygina
e4f1094c50 MDEV-7468 Fix for crash caused by connect.json_udf test
It was allowed to push UDF functions and that caused a crash.
To fix it it was forbidden to push UDF functions from HAVING into WHERE.
2019-02-18 20:37:23 +03:00
Galina Shalygina
7a77b221f1 MDEV-7486: Condition pushdown from HAVING into WHERE
Condition can be pushed from the HAVING clause into the WHERE clause
if it depends only on the fields that are used in the GROUP BY list
or depends on the fields that are equal to grouping fields.
Aggregate functions can't be pushed down.

How the pushdown is performed on the example:

SELECT t1.a,MAX(t1.b)
FROM t1
GROUP BY t1.a
HAVING (t1.a>2) AND (MAX(c)>12);

=>

SELECT t1.a,MAX(t1.b)
FROM t1
WHERE (t1.a>2)
GROUP BY t1.a
HAVING (MAX(c)>12);

The implementation scheme:

1. Extract the most restrictive condition cond from the HAVING clause of
   the select that depends only on the fields that are used in the GROUP BY
   list of the select (directly or indirectly through equalities)
2. Save cond as a condition that can be pushed into the WHERE clause
   of the select
3. Remove cond from the HAVING clause if it is possible

The optimization is implemented in the function
st_select_lex::pushdown_from_having_into_where().

New test file having_cond_pushdown.test is created.
2019-02-17 23:38:44 -08:00
Oleksandr Byelkin
65c5ef9b49 dirty merge 2019-02-07 13:59:31 +01:00
Marko Mäkelä
081fd8bfa2 Merge 10.1 into 10.2 2019-02-02 11:40:02 +02:00
Andrei Elkin
5d48ea7d07 MDEV-10963 Fragmented BINLOG query
The problem was originally stated in
  http://bugs.mysql.com/bug.php?id=82212
The size of an base64-encoded Rows_log_event exceeds its
vanilla byte representation in 4/3 times.
When a binlogged event size is about 1GB mysqlbinlog generates
a BINLOG query that can't be send out due to its size.

It is fixed with fragmenting the BINLOG argument C-string into
(approximate) halves when the base64 encoded event is over 1GB size.
The mysqlbinlog in such case puts out

    SET @binlog_fragment_0='base64-encoded-fragment_0';
    SET @binlog_fragment_1='base64-encoded-fragment_1';
    BINLOG @binlog_fragment_0, @binlog_fragment_1;

to represent a big BINLOG.
For prompt memory release BINLOG handler is made to reset the BINLOG argument
user variables in the middle of processing, as if @binlog_fragment_{0,1} = NULL
is assigned.

Notice the 2 fragments are enough, though the client and server still may
need to tweak their @@max_allowed_packet to satisfy to the fragment
size (which they would have to do anyway with greater number of
fragments, should that be desired).

On the lower level the following changes are made:

Log_event::print_base64()
  remains to call encoder and store the encoded data into a cache but
  now *without* doing any formatting. The latter is left for time
  when the cache is copied to an output file (e.g mysqlbinlog output).
  No formatting behavior is also reflected by the change in the meaning
  of the last argument which specifies whether to cache the encoded data.

Rows_log_event::print_helper()
  is made to invoke a specialized fragmented cache-to-file copying function
  which is

copy_cache_to_file_wrapped()
  that takes care of fragmenting also optionally wraps encoded
  strings (fragments) into SQL stanzas.

my_b_copy_to_file()
  is refactored to into my_b_copy_all_to_file(). The former function
  is generalized
  to accepts more a limit argument to constraint the copying and does
  not reinitialize anymore the cache into reading mode.
  The limit does not do any effect on the fully read cache.
2019-01-24 20:44:50 +02:00
Marko Mäkelä
b5763ecd01 Merge 10.3 into 10.4 2018-12-18 11:33:53 +02:00
Alexander Barkov
34eb98387f MDEV-13995 MAX(timestamp) returns a wrong result near DST change 2018-12-10 19:25:12 +04:00
Alexander Barkov
07e4853c23 MDEV-17563 Different results using table or view when comparing values of time type
MDEV-17625 Different warnings when comparing a garbage to DATETIME vs TIME

- Splitting processes of data type conversion (to TIME/DATE,DATETIME)
  and warning generation.
  Warning are now only get collected during conversion (in an "int" variable),
  and are pushed in the very end of conversion (not in parallel).
  Warnings generated by the low level routines str_to_xxx() and number_to_xxx()
  can now be changed at the end, when TIME_FUZZY_DATES is applied,
  from "Invalid value" to "Truncated invalid value".

  Now "Illegal value" is issued only when the low level routine returned
  an error and TIME_FUZZY_DATES was not set. Otherwise, if the low level
  routine returned "false" (success), or if NULL was converted to a zero
  datetime by TIME_FUZZY_DATES, then "Truncated illegal value"
  is issued. This gives better warnings.

- Methods Type_handler::Item_get_date() and
  Type_handler::Item_func_hybrid_field_type_get_date() now only
  convert and collect warning information, but do not push warnings.

- Changing the return data type for Type_handler::Item_get_date()
  and Type_handler::Item_func_hybrid_field_type_get_date() from
  "bool" to "void". The conversion result (success vs error) can be
  checked by testing ltime->time_type. MYSQL_TIME_{NONE|ERROR}
  mean mean error, other values mean success.

- Adding new wrapper methods Type_handler::Item_get_date_with_warn() and
  Type_handler::Item_func_hybrid_field_type_get_date_with_warn()
  to do conversion followed by raising warnings, and changing
  the code to call new Type_handler::***_with_warn() methods.

- Adding a helper class Temporal::Status, a wrapper
  for MYSQL_TIME_STATUS with automatic initialization.

- Adding a helper class Temporal::Warn, to collect warnings
  but without actually raising them. Moving a part of ErrConv
  into a separate class ErrBuff, and deriving both Temporal::Warn
  and ErrConv from ErrBuff. The ErrBuff part of Temporal::Warn
  is used to collect textual representation of the input data.

- Adding a helper class Temporal::Warn_push. It's used
  to collect warning information during conversion, and
  automatically pushes warnings to the diagnostics area
  on its destructor time (in case of non-zero warning).

- Moving more code from various functions inside class Temporal.

- Adding more Temporal_hybrid constructors and
  protected Temporal methods make_from_xxx(),
  which convert and only collect warning information, but do not
  actually raise warnings.

- Now the low level functions  str_to_datetime() and str_to_time()
  always set status->warning if the return value is "true" (error).

- Now the low level functions number_to_time() and number_to_datetime()
  set the "*was_cut" argument if the return value is "true" (error).

- Adding a few DBUG_ASSERTs to make sure that str_to_xxx() and
  number_to_xxx() always set warnings on error.

- Adding new warning flags MYSQL_TIME_WARN_EDOM and MYSQL_TIME_WARN_ZERO_DATE
  for the code symmetry. Before this change there was a special
  code path for (rc==true && was_cut==0) which was treated by
  Field_temporal::store_invalid_with_warning as "zero date violation".
  Now was_cut==0 always means that there are no any error/warnings/notes
  to be raised, not matter what rc is.

- Using new Temporal_hybrid constructors in combination with
  Temporal::Warn_push inside str_to_datetime_with_warn(),
  double_to_datetime_with_warn(), int_to_datetime_with_warn(),
  Field::get_date(), Item::get_date_from_string(), and a few other places.

- Removing methods Dec_ptr::to_datetime_with_warn(),
  Year::to_time_with_warn(), my_decimal::to_datetime_with_warn(),
  Dec_ptr::to_datetime_with_warn().
  Fixing Sec6::to_time() and Sec6::to_datetime() to
  convert and only collect warnings, without raising warnings.
  Now warning raising functionality resides in Temporal::Warn_push.

- Adding classes Longlong_hybrid_null and Double_null, to
  return both value and the "IS NULL" flag. Adding methods
  Item::to_double_null(), to_longlong_hybrid_null(),
  Item_func_hybrid_field_type::to_longlong_hybrid_null_op(),
  Item_func_hybrid_field_type::to_double_null_op().
  Removing separate classes VInt and VInt_op, as they
  have been replaced by a single class Longlong_hybrid_null.

- Adding a helper method Temporal::type_name_by_timestamp_type(),
  moving a part of make_truncated_value_warning() into it,
  and reusing in Temporal::Warn::push_conversion_warnings().

- Removing Item::make_zero_date() and
  Item_func_hybrid_field_type::make_zero_mysql_time().
  They provided duplicate functionality.
  Now this code resides in Temporal::make_fuzzy_date().
  The latter is now called for all Item types when data type
  conversion (to DATE/TIME/DATETIME) is involved, including
  Item_field and Item_direct_view_ref.
  This fixes MDEV-17563: Item_direct_view_ref now correctly converts
  NULL to a zero date when TIME_FUZZY_DATES says so.
2018-11-08 09:31:46 +04:00
Alexander Barkov
563efeceec MDEV-17607 DATE(COALESCE(year_column)) returns a wrong result
C++ does not guarantee the order of parameter evaluation.
It was wrong to pass item->val_int() and item->null_value
at the same time to any function or constructor.
Adding a new helper class Longlong_null, and new methods
Item::to_longlong_null() and Item_func_hybrid_field_type::to_longlong_null_op(),
which make sure to properly call val_int()/int_op() and test null_value.
Reorganizing the rest of the code accordingly.
2018-11-04 07:13:07 +04:00
Marko Mäkelä
d88c136b9f Merge 10.3 into 10.4 2018-10-17 19:11:42 +03:00
Alexander Barkov
4de0d920be MDEV-17411 Wrong WHERE optimization with simple CASE and searched CASE 2018-10-11 13:39:53 +04:00
Marko Mäkelä
444c380ceb Merge 10.3 into 10.4 2018-10-05 08:09:49 +03:00
Sergei Golubchik
57e0da50bb Merge branch '10.2' into 10.3 2018-09-28 16:37:06 +02:00
Alexander Barkov
ad8e02ac45 MDEV-17317 Add THD* parameter into Item::get_date() and stricter data type control to "fuzzydate" 2018-09-28 14:01:17 +04:00
Alexander Barkov
25ad38abe5 MDEV-17288 Replace Item_func::get_arg0_date() to Date/Datetime methods 2018-09-25 10:07:45 +04:00
Sergei Golubchik
5ae8fce50b Merge branch '10.1' into 10.2 2018-09-24 11:46:08 +02:00
Sergei Golubchik
1fc5a6f30c Merge branch '10.0' into 10.1 2018-09-23 12:58:11 +02:00
Alexander Barkov
80bcb05b24 Merge remote-tracking branch 'origin/5.5' into 10.0 2018-09-21 08:37:42 +04:00
Alexander Barkov
0c6455aa46 MDEV-17249 MAKETIME(-1e50,0,0) returns a wrong result 2018-09-20 16:02:58 +04:00
Marko Mäkelä
734db318ac Merge 10.3 into 10.4 2018-08-16 10:08:30 +03:00
Varun Gupta
befc09f002 MDEV-16722: Assertion `type() != NULL_ITEM' failed
We hit this assert during the create of a temporary table field
because the current code does not handle the case when the value
of the NAME_CONST function is NULL.
Fixed this by allowing creation of temporary table fields even
for the case when NAME_CONST returns NULL value.
Introduced tmp_table_field_from_field_type_maybe_null() function
in Item class so both Item_basic_value and Item_name_const can use it.
Introduced a virtual method get_func_item() in the Item class.
2018-08-12 12:09:56 +05:30
Alexander Barkov
522cd3c7aa Fixing compilation failure on Windows (cannot compare "bool" and "my_bool" directly, cast needed) 2018-08-10 06:22:13 +04:00
Alexander Barkov
cb7b5fbf1c MDEV-16910 Add class VDec
Adding classes VDec and VDec2_lazy, according to the task description.
This patch removes around 250 duplicate code lines.
2018-08-07 10:48:42 +04:00
Marko Mäkelä
05459706f2 Merge 10.2 into 10.3 2018-08-03 15:57:23 +03:00
Alexander Barkov
c7115428ed MDEV-16881 Remove Item::get_temporal_with_sql_mode() and val_xxx_from_date() 2018-08-02 15:36:13 +04:00
Alexander Barkov
54fc47960d MDEV-16874 Implement class Item_handled_func 2018-08-01 16:15:16 +04:00
Alexander Barkov
5c5a116b47 MDEV-16614 signal 7 after calling stored procedure, that uses regexp
The problem happened in the derived condition pushdown code:
- When Item_func_regex::build_clone() was called, it created a copy of
  the original Item_func_regex, and this copy got registered in free_list.
  Class specific additional dynamic members (such as "re") made
  a shallow copy, rather than a deep copy, in the cloned Item_func_regex.
  As a result, the Regexp_processor_pcre::m_pcre of the cloned Item_func_regex
  and of the original Item_func_regex pointed to the same compiled regular
  expression.
- On cleanup_items(), both original and cloned copies of Item_func_regex
  called re.cleanup(), which called pcre_free(m_pcre). So the same compiled
  regular expression was freed two times, which was noticed by ASAN.

The same problem was repeatable for Item_func_regexp_instr.

A similar problem happened for Item_func_sp, for the sp_result_field member.
Both original and cloned copies of Item_func_sp pointed the same Field instance
and both deleted it on cleanup().

A possible solution would be to fix build_clone() to create deep
(instead of shallow) copies for the dynamic members of the affected classes
(Item_func_regex, Item_func_regexp_instr, Item_func sp).
However, this would be too complex.

As agreed with Galina and Igor, this patch disallows using using these
affected classes in derived condition pushdown by overriding get_clone()
to return NULL.
2018-07-26 13:05:22 +04:00
Alexander Barkov
fee4632387 MDEV-15758 Split Item_bool_func::get_mm_leaf() into virtual methods in Field and Type_handler 2018-07-20 11:07:19 +04:00
Alexander Barkov
e61568ee93 Merge remote-tracking branch 'origin/10.3' into 10.4 2018-07-03 14:02:05 +04:00
Alexander Barkov
e213b20e07 MDEV-16592 Change Item::with_sum_func from a member to a virtual method 2018-06-27 14:48:03 +04:00
Oleksandr Byelkin
083279f783 Merge commit '6b8802e8dd5467556a024d807a1df23940b00895' into bb-10.3-fix_len_dec 2018-06-19 14:51:50 +02:00
Oleksandr Byelkin
6b8802e8dd MDEV-11071: Assertion `thd->transaction.stmt.is_empty()' failed in Locked_tables_list::unlock_locked_table
fix_length_and_dec now return result (error/OK)
2018-06-15 10:31:30 +02:00
Alexander Barkov
c20cd68e60 MDEV-14630 Replace {STRING|INT|REAL|DECIMAL|DATE}_ITEM to CONST_ITEM 2018-06-06 14:09:06 +04:00
Alexander Barkov
f4dfc609cf MDEV-16388 Replace member Item::fixed to virtual method is_fixed() 2018-06-05 11:56:19 +04:00
Igor Babaev
cab1d63826 Merge branch '10.3' into 10.4 2018-06-03 10:34:41 -07:00
Alexander Barkov
637af78383 MDEV-16309 Split ::create_tmp_field() into virtual methods in Item
Detailed: changes:
1. Moving Field specific code into new methods on Field:
   - Field *Field::create_tmp_field(...)
   - virtual void init_for_tmp_table(...)

2. Removing virtual Item::create_tmp_field().
   Adding instead a new virtual method Item::create_tmp_field_ex().

   Note, a virtual create_tmp_field() still exists, but only for Item_sum.
   This resembles 10.0 code structure. Perhaps create_tmp_field() should
   be removed from Item_sum and Item_sum descendants should override
   create_tmp_field_ex() directly. This can be done in a separate commit.

3. Adding helper classes Tmp_field_src and Tmp_field_param,
   to make the API for Item::create_tmp_field_ex() smaller
   and easier to extend in the future.

4. Decomposing the public function create_tmp_field() into
   virtual implementations for Item and a number of its descendants:
   - Item_basic_value
   - Item_sp_variable
   - Item_name_const
   - Item_result_field
   - Item_field
   - Item_ref
   - Item_type_holder
   - Item_row
   - Item_func_sp
   - Item_func_user_var
   - Item_sum
   - Item_sum_field
   - Item_proc

5. Adding DBUG_ASSERT-only virtual implementations for
   Item types that should not appear in create_tmp_table_ex(),
   for easier debugging:
   - Item_nodeset_func
   - Item_nodeset_to_const_comparator
   - Item_null_result
   - Item_copy
   - Item_ident_for_show
   - Item_user_var_as_out_param

6. Moving public function create_tmp_field_from_field()
   as a method to Item_field.

7. Removing Item::set_result_field(). It's not needed any more.

8. Cleanup: Removing the enum value "EXPR_CACHE_ITEM",
   as it's not used for a very long time.
2018-05-28 16:57:59 +04:00
Sergey Vojtovich
aeaac6ca76 moved include <cmath> from my_global.h 2018-05-26 13:31:12 +04:00
Sergey Vojtovich
54999f4e75 Use std::isfinite in C++ code
This is addition to parent revision fixing build failures.
2018-05-24 00:12:25 +04:00
Sergei Golubchik
28dbdf3d79 MDEV-14551 Can't find record in table on multi-table update with ORDER BY
preserve positions if the multi-update join is using tmp table:

* store positions in the tmp table if needed
  JOIN::add_fields_for_current_rowid()

* take positions from the tmp table, not from file->position():
  multi_update::prepare2()
2018-05-17 15:13:47 +02:00
Galina Shalygina
d3ff133390 MDEV-12387 Push conditions into materialized subqueries
The logic and the implementation scheme are similar with the
MDEV-9197 Pushdown conditions into non-mergeable views/derived tables

How the push down is made on the example:

select * from t1
where a>3 and b>10 and
 (a,b) in (select x,max(y) from t2 group by x);

-->

select * from t1
where a>3 and b>10 and
  (a,b) in (select x,max(y)
            from t2
            where x>3
            group by x
            having max(y)>10);

The implementation scheme:

1. Search for the condition cond that depends only on the fields
   from the left part of the IN subquery (left_part)
2. Find fields F_group in the select of the right part of the
   IN subquery (right_part) that are used in the GROUP BY
3. Extract from the cond condition cond_where that depends only on the
   fields from the left_part that stay at the same places in the left_part
   (have the same indexes) as the F_group fields in the projection of the
   right_part
4. Transform cond_where so it can be pushed into the WHERE clause of the
   right_part and delete cond_where from the cond
5. Transform cond so it can be pushed into the HAVING clause of the right_part

The optimization is made in the
Item_in_subselect::pushdown_cond_for_in_subquery() and is controlled by the
variable condition_pushdown_for_subquery.

New test file in_subq_cond_pushdown.test is created.

There are also some changes made for setup_jtbm_semi_joins().
Now it is decomposed into the 2 procedures: setup_degenerate_jtbm_semi_joins()
that is called before optimize_cond() for cond and setup_jtbm_semi_joins()
that is called after optimize_cond().
New setup_jtbm_semi_joins() is made in the way so that the result of its work is
the same as if it was called before optimize_cond().

The code that is common for pushdown into materialized derived and into materialized
IN subqueries is factored out into pushdown_cond_for_derived(),
Item_in_subselect::pushdown_cond_for_in_subquery() and
st_select_lex::pushdown_cond_into_where_clause().
2018-05-15 23:45:59 +02:00
Oleksandr Byelkin
0dd1ebcb27 MDEV-15576: Server crashed in Cached_item_str::cmp / sortcmp or Assertion `item->null_value' failed in Type_handler_temporal_result::make_sort_key upon SELECT with NULLIF and ROLLUP
Fixed null_value processing and is_null() usage.
2018-05-15 14:20:54 +02:00
Sergei Golubchik
c9717dc019 Merge branch '10.2' into 10.3 2018-05-11 13:15:10 +02:00
Sergei Golubchik
9b1824dcd2 Merge branch '10.1' into 10.2 2018-05-10 13:01:42 +02:00
Sergei Golubchik
9989c26bc9 Merge branch '10.0' into 10.1 2018-05-05 14:01:59 +02:00
Sergei Golubchik
c4499a0391 Merge branch '5.5' into 10.0 2018-04-29 00:38:10 +02:00
Igor Babaev
eb057dce20 MDEV-15035 Wrong results when calling a stored procedure
multiple times with different arguments.

If the ON expression of an outer join is an OR formula with one
of the disjunct being a constant formula then the expression
cannot be null-rejected if the constant formula is true. Otherwise
it can be null-rejected and if so the outer join can be converted
into inner join. This optimization was added in the patch for
mdev-4817. Yet the code had a defect: if the query was used in
a stored procedure with parameters and the constant item contained
some of them then the value of this constant item depended on the
values of the parameters. With some parameters it may be true,
for others not. The validity of conversion to inner join is checked
only once and it happens only for the first call of procedure.
So if the  parameters in the first call allowed the conversion it
was done and next calls used the transformed query though there
could be calls whose parameters made the conversion invalid.

Fixed by cheking whether the constant disjunct in the ON expression
originally contained an SP parameter. If so the expression is not
considered as null-rejected. For this check a new item's attribute
was intruduced: Item::with_param. It is calculated for each item
by fix fields() functions.
Also moved the call of optimize_constant_subqueries() in
JOIN::optimize after the call of simplify_joins(). The reason
for this is that after the optimization introduced by the patch
for mdev-4817 simplify_joins() can use the results of execution
of non-expensive constant subqueries and this is not valid.
2018-04-25 09:22:06 -07:00
Marko Mäkelä
7396dfcca7 Merge 10.2 into 10.3 2018-04-24 20:59:57 +03:00
Marko Mäkelä
4cd7979c56 Merge 10.1 into 10.2 2018-04-24 09:39:45 +03:00