Commit graph

6954 commits

Author SHA1 Message Date
Sergei Golubchik
c93ac0a1c6 cleanups and simplifications 2015-10-05 17:14:15 +02:00
Sergei Golubchik
7ca8b4bbfa move internal API out from group_by_handler
into a Pushdown_query object
2015-10-05 17:14:15 +02:00
Sergei Golubchik
e8daa41885 typos in comments, minor stylistic edits 2015-10-05 17:14:14 +02:00
Monty
cf50e13fbd MDEV-6080: Allowing storage engine to shortcut group by queries
This task is to allow storage engines that can execute GROUP BY or
summary queries efficiently to intercept a full query or sub query from
MariaDB and deliver the result either to the client or to a temporary
table for further processing.

- Added code in sql_select.cc to intercept GROUP BY queries.
  Creation of group_by_handler is done after all optimizations to allow
  storage engine to benefit of an optimized WHERE clause and suggested
  indexes to use.
- Added group by handler to sequence engine and a group_by test suite as
  a way to test the new interface.
- Intercept EXPLAIN with a message "Storage engine handles GROUP BY"

libmysqld/CMakeLists.txt:
  Added new group_by_handler files
sql/CMakeLists.txt:
  Added new group_by_handler files
sql/group_by_handler.cc:
  Implementation of group_by_handler functions
sql/group_by_handler.h:
  Definition of group_by_handler class
sql/handler.h:
  Added handlerton function to create a group_by_handler, if the storage
  engine can intercept the query.
sql/item_cmpfunc.cc:
  Allow one to evaluate item_equal any time.
sql/sql_select.cc:
  Added code to intercept GROUP BY queries
  - If all tables are from the same storage engine and the query is
    using sum functions, call create_group_by() to check if the storage
    engine can intercept the query.
    - If yes:
       - create a temporary table to hold a GROUP_BY row or result
       - In do_select() intercept normal query execution by instead
         calling the group_by_handler to get the result
       - Intercept EXPLAIN
sql/sql_select.h:
  Added handling of group_by_handler
  Added caching of the original join tab (needed for cleanup after
  group_by handler)
storage/sequence/mysql-test/sequence/group_by.result:
  Test group_by_handler interface
storage/sequence/mysql-test/sequence/group_by.test:
  Test group_by_handler interface
storage/sequence/sequence.cc:
  Added simple group_by_engine for handling COUNT(*) and
  SUM(primary_key).  This was done as a test of the group_by_handler
  interface
2015-10-05 17:14:14 +02:00
Monty
d8df2b9464 Bugs, found by valgrind 2015-10-05 17:14:14 +02:00
Monty
ae2cdc1798 fix comments 2015-10-05 17:14:14 +02:00
Monty
8ed5fde3e1 bug in JOIN_TAB::cleanup() that caused freed memory to be accessed 2015-10-05 17:14:14 +02:00
Sreeharsha Ramanavarapu
415faa122b Bug #19434916: FATAL_SIGNAL IN ADD_KEY_EQUAL_FIELDS() WITH
UPDATE VIEW USING OUTER SUBQUERY

Issue:
-----
While resolving a column which refers to a table/view in an
outer query, it's respecitve item object is marked with the
outer query's select_lex object. But when the column refers
to a view or if the column is part of a subquery in the
HAVING clause, an Item_ref object is created. While the
reference to the outer query is stored by the Item_ref
object, the same is not stored in it's real_item.

This creates a problem with the IN-TO-EXISTS optmization.
When there is an index over the column in the inner query,
it will be considered since the column's real_item object
will be mistaken for a local field. This will lead to a
crash.

SOLUTION:
---------
Under the current design, the only way to fix this issue is
to check the reginfo.join_tab for a NULL value. If yes, the
query should not be worrying about the key use.

The testcase and comments added as part of the fix for
Bug#17766653 have been backported.
2015-10-01 07:45:27 +05:30
Sergei Petrunia
7016621596 MDEV-8829: Assertion `0' failed in Explain_table_access::tag_to_json
- Add EXPLAIN/ANALYZE FORMAT=JSON handling for a few special cases.
2015-09-24 15:45:54 +03:00
Alexander Barkov
c8db0df2da Removing "DTCollation Arg_comparator::cmp_collation".
Using a CHARSET_INFO pointer instead.
2015-09-21 16:18:20 +04:00
Alexander Barkov
afa1773439 Moving Item_args::arg_count from "public" to "protected". 2015-09-21 12:40:07 +04:00
Oleksandr Byelkin
da3ec3d421 MDEV-7970: EXPLAIN FORMAT=JSON does not print HAVING
Printing non-trivial HAVING added.
2015-09-18 16:08:13 +02:00
Alexander Barkov
79140b0383 MDEV-8793 Wrong result set for SELECT ...
WHERE COALESCE(time_column)=TIME('00:00:00')
            AND COALESCE(time_column)=DATE('2015-09-11')
MDEV-8814 Wrong result for WHERE datetime_column > TIME('00:00:00')
2015-09-18 13:30:44 +04:00
Alexander Barkov
3a0df3cf3c MDEV-8372 Use helper methods introduced in MDEV-7824 all around the code 2015-09-15 10:11:52 +04:00
Alexander Barkov
0302efca7f MDEV-8705 Wrong result for SELECT..WHERE latin1_bin_column='a' AND latin1_bin_column='A'
MDEV-8712 Wrong result for SELECT..WHERE latin1_bin_column=_latin1'a' AND latin1_bin_column='A'
2015-09-11 09:20:40 +04:00
Alexander Barkov
39b46ae934 MDEV-8706 Wrong result for SELECT..WHERE time_column=TIMESTAMP'2015-08-30 00:00:00' AND time_column='00:00:00' 2015-09-09 15:39:09 +04:00
Alexander Barkov
c108019a48 MDEV-8747 Wrong result for SELECT..WHERE derived_table_column='a' AND derived_table_column<>_latin1'A' COLLATE latin1_bin
MDEV-8749 Wrong result for SELECT..WHERE derived_table_enum_column='number' AND derived_table_enum_column OP number
2015-09-06 01:30:46 +04:00
Sergei Golubchik
530a6e7481 Merge branch '10.0' into 10.1
referenced_by_foreign_key2(), needed for InnoDB to compile,
was taken from 10.0-galera
2015-09-03 12:58:41 +02:00
Monty
56aa19989f MDEV-6152: Remove calls to current_thd while creating Item
Part 5: Removing calls to current_thd in net_read calls, creating fields,
        query_cache, acl and some other places where thd was available
2015-09-01 18:42:02 +03:00
Alexander Barkov
8ea9b8c0b1 MDEV-8722 The patch for MDEV-8688 disabled equal field propagation for temporal column and BETWEEN and IN
Item::cmp_context was inconsistently used in combination with cmp_type()
and result_type() in different places of the code. Fixed to use cmp_type()
in all places where cmp_context is involved, to avoid unexpected results
for temporal data types (which have result_type()==STRING_RESULT and
cmp_type==TIME_RESULT).
2015-09-01 19:24:58 +04:00
Alexander Barkov
a3c24ee739 MDEV-8707 Wrong result for
SELECT..WHERE varchar_column=DATE'2001-01-01' AND varchar_column='2001-01-01'
2015-08-31 18:18:10 +04:00
Monty
3bca8db4f9 MDEV-6152: Remove calls to current_thd while creating Item
- Part 4: Removing calls to sql_alloc() and sql_calloc()

Other things:
- Added current_thd in some places to make it clear that it's called (easier to remove later)
- Move memory allocation from Item_func_case::fix_length_and_dec() to Item_func_case::fix_fields()
- Added mem_root to some new calls
- Fixed some wrong UNINIT_VAR() calls
- Fixed a bug in generate_partition_syntax() in case of errors
- Added mem_root to argument to new thread_info
- Simplified my_parse_error() call in sql_yacc.yy
2015-08-27 22:29:11 +03:00
Monty
3cb578c001 MDEV-6152: Remove calls to current_thd while creating Item
- Part 3: Adding mem_root to push_back() and push_front()

Other things:
- Added THD as an argument to some partition functions.
- Added memory overflow checking for XML tag's in read_xml()
2015-08-27 22:21:08 +03:00
Alexander Barkov
1b6b44b6b5 MDEV-8661 Wrong result for SELECT..WHERE a='a' AND a='a' COLLATE latin1_bin
MDEV-8679 Equal field propagation is not used for VARCHAR when it safely could
2015-08-26 22:32:01 +04:00
Alexander Barkov
98ba2bf424 Clean-up: moving compare_collation() from Item to Item_bool_func. 2015-08-21 17:08:55 +04:00
Monty
1bae0d9e56 Stage 2 of MDEV-6152:
- Added mem_root to all calls to new Item
- Added private method operator new(size_t size) to Item to ensure that
  we always use a mem_root when creating an item.

This saves use once call to current_thd per Item creation
2015-08-21 10:40:51 +04:00
Sergey Vojtovich
31e365efae MDEV-8010 - Avoid sql_alloc() in Items (Patch #1)
Added mandatory thd parameter to Item (and all derivative classes) constructor.
Added thd parameter to all routines that may create items.
Also removed "current_thd" from Item::Item. This reduced number of
pthread_getspecific() calls from 290 to 177 per OLTP RO transaction.
2015-08-21 10:40:39 +04:00
Sergei Petrunia
9b475ee3c1 MDEV-8289: Semijoin inflates number of rows in query result
- Make semi-join optimizer not to choose LooseScan
  when 1) the index is not covered and 2) full index
  scan will be required.

- Make sure that the code in make_join_select() that may change
  full index scan into a range scan is not invoked when the table
  uses full scan.
2015-08-18 22:54:42 +03:00
Ajo Robert
552b1c8ab6 Merge branch 'mysql-5.1' into mysql-5.5 2015-08-07 16:27:48 +05:30
Ajo Robert
f3dce250f4 Bug #20760261 mysqld crashed in materialized_cursor::
send_result_set_metadata

Analysis
--------
Cursor inside trigger accessing NEW/OLD row leads server exit.

The reason for the bug was that implementation of function
create_tmp_table() was not considering Item::TRIGGER_FIELD_ITEM
as possible alternative for type of class being instantiated.
This was resulting in a mismatch between a number of columns
in result list and temp table definition. This mismatch leads
to the failure of assertion
DBUG_ASSERT(send_result_set_metadata.elements == item_list.elements)
in the method Materialized_cursor::send_result_set_metadata
in debug mode.

Fix:
---
Added code to consider Item::TRIGGER_FIELD_ITEM as valid
type while creating fields.
2015-08-07 16:26:10 +05:30
Mithun C Y
c20911dbe0 Merge branch 'mysql-5.1' into mysql-5.5 2015-08-04 12:28:56 +05:30
Mithun C Y
c28626d0af Bug #21096444: MYSQL IS TRYING TO PERFORM A CONSISTENT READ BUT THE READ VIEW IS NOT ASSIGNED!
Issue: A select for update subquery in having clause
resulted deadlock and its transaction was rolled back
by innodb. val_XXX interfaces do not handle errors and
it do not propogate errors to its caller. sub_select
did not see this error when it called
evaluate_join_record and later made a call to innodb.
As transaction is rolled back innodb asserted.

Fix: Now evaluate_join_record checks if there is any
error reported and then return the same to its caller.
2015-08-04 11:45:02 +05:30
Jan Lindström
9a5787db51 Merge commit '96badb16afcf' into 10.0
Conflicts:
	client/mysql_upgrade.c
	mysql-test/r/func_misc.result
	mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
	mysql-test/suite/innodb/r/innodb-fk.result
	mysql-test/t/subselect_sj_mat.test
	sql/item.cc
	sql/item_func.cc
	sql/log.cc
	sql/log_event.cc
	sql/rpl_utility.cc
	sql/slave.cc
	sql/sql_class.cc
	sql/sql_class.h
	sql/sql_select.cc
	storage/innobase/dict/dict0crea.c
	storage/innobase/dict/dict0dict.c
	storage/innobase/handler/ha_innodb.cc
	storage/xtradb/dict/dict0crea.c
	storage/xtradb/dict/dict0dict.c
	storage/xtradb/handler/ha_innodb.cc
	vio/viosslfactories.c
2015-08-03 23:09:43 +03:00
Monty
9bb8b74e99 Merge branch '10.1' of github.com:MariaDB/server into 10.1
Conflicts:
	sql/item_subselect.cc

Fixed also typo in comment
2015-07-09 14:47:32 +03:00
Monty
7332af49e4 - Renaming variables so that they don't shadow others (After this patch one can compile with -Wshadow and get much fewer warnings)
- Changed ER(ER_...) to ER_THD(thd, ER_...) when thd was known or if there was many calls to current_thd in the same function.
- Changed ER(ER_..) to ER_THD_OR_DEFAULT(current_thd, ER...) in some places where current_thd is not necessary defined.
- Removing calls to current_thd when we have access to thd

Part of this is optimization (not calling current_thd when not needed),
but part is bug fixing for error condition when current_thd is not defined
(For example on startup and end of mysqld)

Notable renames done as otherwise a lot of functions would have to be changed:
- In JOIN structure renamed:
   examined_rows -> join_examined_rows
   record_count -> join_record_count
- In Field, renamed new_field() to make_new_field()

Other things:
- Added DBUG_ASSERT(thd == tmp_thd) in Item_singlerow_subselect() just to be safe.
- Removed old 'tab' prefix in JOIN_TAB::save_explain_data() and use members directly
- Added 'thd' as argument to a few functions to avoid calling current_thd.
2015-07-06 20:24:14 +03:00
Monty
86377d078e Fixes done while working on MDEV-4119:
Fixed several optimizer issues relatied to GROUP BY:

a) Refering to a SELECT column in HAVING sometimes calculated it twice, which caused problems with non determinstic functions

b) Removing duplicate fields and constants from GROUP BY was done too late for "using index for group by" optimization to work

c) EXPLAIN SELECT ... GROUP BY did wrongly show 'Using filesort' in some cases involving "Using index for group-by"


a) was fixed by:
- Changed last argument to Item::split_sum_func2() from bool to int to allow more flags
- Added flag argument to Item::split_sum_func() to allow on to specify if the item was in the SELECT part
- Mark all split_sum_func() calls from SELECT with SPLIT_SUM_SELECT
- Changed split_sum_func2() to do nothing if called with an argument that is not a sum function and doesn't include sum functions, if we are not an argument to SELECT.

This ensures that in a case like
select a*sum(b) as f1 from t1 where a=1 group by c having f1 <= 10;

That 'a' in the SELECT part is stored as a reference in the temporary table togeher with sum(b) while the 'a' in having isn't (not needed as 'a' is already a reference to a column in the result)

b) was fixed by:
- Added an extra remove_const() pass for GROUP BY arguments before make_join_statistics() in case of one table SELECT.

This allowes get_best_group_min_max() to optimize things better.

c) was fixed by:
- Added test for group by optimization in JOIN::exec_inner for
  select->quick->get_type() == QUICK_SELECT_I::QS_TYPE_GROUP_MIN_MAX

item.cc:
- Simplifed Item::split_sum_func2()
  - Split test to make them faster and easier to read
  - Changed last argument to Item::split_sum_func2() from bool to int to allow more flags
  - Added flag argument to Item::split_sum_func() to allow on to specify if the item was in the SELECT part
  - Changed split_sum_func2() to do nothing if called with an argument that is not a sum function and doesn't include sum functions, if we are not an argument to SELECT.

opt_range.cc:
- Simplified get_best_group_min_max() by calcuating first how many group_by elements.
- Use join->group instead of join->group_list to test if group by, as join->group_list may be NULL if everything was optimized away.

sql_select.cc:
- Added an extra remove_const() pass for GROUP BY arguments before make_join_statistics() in case of one table SELECT.
- Use group instead of group_list to test if group by, as group_list may be NULL if everything was optimized away.
- Moved printing of "Error in remove_const" to remove_const() instead of having it in caller.
- Simplified some if tests by re-ordering code.
- update_depend_map_for_order() and remove_const() fixed to handle the case where make_join_statistics() has not yet been called (join->join_tab is 0 in this case)
2015-07-05 12:39:46 +03:00
Alexander Barkov
768620ee5c MDEV-8189 field<>const and const<>field are not symmetric 2015-06-30 12:56:31 +04:00
Alexander Barkov
4d4f2ed2e1 Moving Item_bool_func::add_key_fields_optimize_op() to Item_bool_func2.
It's now needed outside of Item_bool_func2 any more.
2015-06-28 18:23:15 +04:00
Sergei Golubchik
e04f6e7e3a Merge branch 'bb-10.1-serg' into 10.1 2015-06-28 15:23:33 +02:00
Alexander Barkov
ee0237fe89 MDEV-8330 Get rid of Item_func::select_optimize() and Item_func::optimize_type 2015-06-28 14:29:51 +04:00
Sergei Golubchik
658992699b Merge tag 'mariadb-10.0.20' into 10.1 2015-06-27 20:35:26 +02:00
Alexander Barkov
a4b0063d7b MDEV-8256 A part of a ROW comparison is erroneously optimized away
Item_func_eq's created during conversion of a ROW equality to a conjunction
of scalar equalities did not set cmp_context for its arguments properly,
so some of these created Item_func_eq could be later erroneously eliminated.
2015-06-26 10:58:51 +04:00
Alexander Barkov
115904c2c8 MDEV-7824 [Bug #68041] Zero date can be inserted in strict no-zero mode through a default value 2015-06-26 10:16:51 +04:00
Monty
8c815751c9 Problem was that for cases like:
SELECT ... WHERE XX IN (SELECT YY)
this was transformed to something like:
SELECT ... WHERE IF_EXISTS(SELECT ... HAVING XX=YY)

The bug was that for normal execution XX was fixed in the original outer SELECT context while in PS it was fixed in the sub query context and this confused the optimizer.

Fixed by ensuring that XX is always fixed in the outer context.
2015-06-25 23:26:29 +03:00
Monty
2e941fe9fc Fixed crashing bug when using ONLY_FULL_GROUP_BY in a stored procedure/trigger that is repeatedly executed.
This is MDEV-7601, including it's sub tasks MDEV-7594, MDEV-7555, MDEV-7590, MDEV-7581, MDEV-7589

The problem was that select_lex->non_agg_fields was not properly reset for re-execution and this caused an overwrite of a random memory position.
The fix was move non_agg_fields from select_lext to JOIN, which is properly reset.
2015-06-25 23:18:48 +03:00
Sergey Vojtovich
9b57b214c4 MDEV-8199 - first_breadth_first_tab() takes 0.07% in OLTP RO
This is an addition to original patch: cleanups for next_breadth_first_tab().

Overhead change:
next_breadth_first_tab()  0.04% -> out of radar
2015-06-23 09:54:36 +04:00
Sergey Vojtovich
45f41b52e8 MDEV-8199 - first_breadth_first_tab() takes 0.07% in OLTP RO
Split first_breadth_first_tab() into
JOIN::first_breadth_first_optimization_tab() and
JOIN::first_breadth_first_execution_tab().

This allows to eliminate function call and one condition. Adjusted callers
accordingly.

Overhead change:
first_breadth_first_tab()        0.07% -> out of radar
next_breadth_first_tab()         0.04% -> 0.04%
JOIN::cleanup()                  0.15% -> 0.11%
JOIN::save_explain_data_intern() 0.28% -> 0.24%
2015-06-23 09:54:36 +04:00
Sergei Petrunia
ebe2bd74fe MDEV-7836: ANALYZE FORMAT=JSON should provide info about GROUP BY
ANALYZE should also record remove_duplicates() operation.
2015-06-20 04:20:18 +03:00
Sergey Vojtovich
2bc6e29afe MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
Pass THD to Sort_and_group_tracker::report_sorting().
This reduces number of pthread_getspecific() calls from 292 to 290.
2015-06-19 15:06:34 +04:00
Sergey Vojtovich
b85e5efca3 MDEV-7943 - pthread_getspecific() takes 0.76% in OLTP RO
Pass THD to find_all_keys() and Item_equal::Item_equal().
In MRR use table->in_use instead of current_thd.

This reduces number of pthread_getspecific() calls from 354 to 320.
2015-06-19 15:06:34 +04:00