Commit graph

318 commits

Author SHA1 Message Date
Sergey Petrunya
4480642624 Added TODO comments 2011-06-27 23:38:56 +04:00
Sergey Petrunya
6adddca80e Make semi-joins work with outer joins part #1:
- Make make_outerjoin_info() correctly process semi-join nests
- Make make_join_select() attach conditions to the right places.
2011-06-22 01:57:28 +04:00
unknown
44570d2b12 MWL#89
Automerged with 5.3.
2011-06-21 16:00:41 +03:00
unknown
a02682abcc MWL#89
- Added regression test with queries over the WORLD database.
- Discovered and fixed several bugs in the related cost calculation
  functionality both in the semijoin and non-semijon subquery code.
- Added DBUG printing of the cost variables used to decide between
  IN-EXISTS and MATERIALIZATION.
2011-06-21 15:50:07 +03:00
Sergey Petrunya
a0973ba22e Merge fix for BUG#778406. 2011-06-17 17:45:41 +04:00
Igor Babaev
078b59f5bc Merge of mwl #106 into 5.3. 2011-06-15 21:48:38 -07:00
Sergey Petrunya
bce51cebcb BUG#778406: Crash in hp_movelink with Aria engine and subqueries
-In  do_sj_dups_weedout(), set nulls_ptr to point to NULL bytes (and not to length bytes) of the DuplicateWeedout column.
2011-06-15 18:37:01 +04:00
Igor Babaev
059aff15cd Fixed LP bug #784441.
The code that added semi-join transformations missed checking
the state of the fixed flag for the items built with the
and_items function before calls of the fix_fields method.
This could lead to an abort failure when the first argument
of and_items() happened to be NULL.
2011-06-06 15:50:46 -07:00
Igor Babaev
f03a3ee54f Merged the code of mwl 106 into the latest 5.3 with mwl 90 pushed.
Resolved all conflicts and failures.
2011-06-04 19:56:06 -07:00
Sergey Petrunya
ebb130b189 BUG#787299: Valgrind complains on a join query with two IN subqueries
- Don't attempt to construct FirstMatch access method if we've 
  just figured three lines above that it can't be used (because join
  prefix doesn't have the needed tables), and so have set 
    pos->first_firstmatch_table= MAX_TABLES 
  Attempts to analyze join->positions[MAX_TABLES] caused valgrind warnings
2011-06-02 23:48:33 +04:00
Sergey Petrunya
0cf912c23f - Remove compiler warning
- Remove garbage comments
2011-05-30 11:19:40 +04:00
Sergey Petrunya
5cd18326c2 Merge 5.3->main -> 5.3-mwl90 2011-05-29 12:58:44 +04:00
Sergey Petrunya
ded7342daa - Update test results (trivial)
- Code cleanup: remove garbage comments
2011-05-29 01:47:24 +04:00
Sergey Petrunya
77c4c4d8ea post-merge fixes: get MWL#90 code to work with MWL#89's
way of processing prepared statements:
- conversion subquery_predicate -> TABLE_LIST is once per-statement
- However, the code must take into account that materialized temptable
  is dropped and re-created on each execution (the tricky part is that 
  at start of n-th EXECUTE we have TABLE_LIST object but not its TABLE object)
- IN-equality is injected into WHERE on every execution.
2011-05-28 20:34:04 +04:00
Michael Widenius
2894d50e3e automatic merge with 5.3 2011-05-28 05:58:16 +03:00
Michael Widenius
f197991f41 Merge with 5.1-microseconds
A lot of small fixes and new test cases.

client/mysqlbinlog.cc:
  Cast removed
client/mysqltest.cc:
  Added missing DBUG_RETURN
include/my_pthread.h:
  set_timespec_time_nsec() now only takes one argument
mysql-test/t/date_formats.test:
  Remove --disable_ps_protocl as now also ps supports microseconds
mysys/my_uuid.c:
  Changed to use my_interval_timer() instead of my_getsystime()
mysys/waiting_threads.c:
  Changed to use my_hrtime()
sql/field.h:
  Added bool special_const_compare() for fields that may convert values before compare (like year)
sql/field_conv.cc:
  Added test to get optimal copying of identical temporal values.
sql/item.cc:
  Return that item_int is equal if it's positive, even if unsigned flag is different.
  Fixed Item_cache_str::save_in_field() to have identical null check as other similar functions
  Added proper NULL check to Item_cache_int::save_in_field()
sql/item_cmpfunc.cc:
  Don't call convert_constant_item() if there is nothing that is worth converting.
  Simplified test when years should be converted
sql/item_sum.cc:
  Mark cache values in Item_sum_hybrid as not constants to ensure they are not replaced by other cache values in compare_datetime()
sql/item_timefunc.cc:
  Changed sec_to_time() to take a my_decimal argument to ensure we don't loose any sub seconds.
  Added Item_temporal_func::get_time() (This simplifies some things)
sql/mysql_priv.h:
  Added Lazy_string_decimal()
sql/mysqld.cc:
  Added my_decimal constants max_seconds_for_time_type, time_second_part_factor
sql/table.cc:
  Changed expr_arena to be of type CONVENTIONAL_EXECUTION to ensure that we don't loose any items that are created by fix_fields()
sql/tztime.cc:
  TIME_to_gmt_sec() now sets *in_dst_time_gap in case of errors
  This is needed to be able to detect if timestamp is 0
storage/maria/lockman.c:
  Changed from my_getsystime() to set_timespec_time_nsec()
storage/maria/ma_loghandler.c:
  Changed from my_getsystime() to my_hrtime()
storage/maria/ma_recovery.c:
  Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/maria/unittest/trnman-t.c:
  Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/xtradb/handler/ha_innodb.cc:
  Added support for new time,datetime and timestamp
unittest/mysys/thr_template.c:
  my_getsystime() -> my_interval_timer()
unittest/mysys/waiting_threads-t.c:
  my_getsystime() -> my_interval_timer()
2011-05-28 05:11:32 +03:00
Sergey Petrunya
aa551f1b9e Post-merge fixes part#2 2011-05-26 15:01:26 +04:00
Sergey Petrunya
77b3b960b1 Merge MWL#90 with 5.3-main 2011-05-25 19:31:13 +04:00
unknown
8a314700f9 MWL#89 automatic merge with 5.3 2011-05-23 11:54:26 +03:00
unknown
742dfc92a2 MWL#89: Address review feedback (by Sergey Petrunia)
mysql-test/r/subselect4.result:
  Moved test case for LP BUG#718593 into the correct test file subselect_mat_cost_bugs.test.
mysql-test/t/subselect4.test:
  Moved test case for LP BUG#718593 into the correct test file subselect_mat_cost_bugs.test.
2011-05-23 10:56:05 +03:00
Igor Babaev
89cf840cd9 Merge 2011-05-20 12:47:39 -07:00
Sergey Petrunya
23cfe1261d Merge fix for BUG#784723 2011-05-20 10:13:02 +04:00
Sergey Petrunya
d1138283b9 BUG#784723: Wrong result with semijoin + nested subqueries in maria-5.3
- in advance_sj_state(), remember join->cur_dups_producing_tables in 
  pos->prefix_dups_producing_tables *before* we modify it, so that 
  restore_prev_sj_state() restores cur_dups_producing_tables in all cases.
- Updated test results in subselect_sj2[_jcl6].result (the original EXPLAIN
  was invalid there)
2011-05-20 01:05:06 +04:00
unknown
56c0e19ed0 MWL#89 Addressing Sergey's review comments - Part 1.
Address the 'trivial' part of Sergey's review of MWL#89.
2011-05-18 01:23:22 +03:00
Igor Babaev
704f97035f Merged the code of MWL#106 into 5.3
Resolved all conflicts, bad merges and fixed a few minor bugs in the code.
Commented out the queries from multi_update, view, subselect_sj, func_str,
derived_view, view_grant that failed either with crashes in ps-protocol or
with wrong results.
The failures are clear indications of some bugs in the code and these bugs
are to be fixed.
2011-05-16 22:39:43 -07:00
unknown
18d08eeacc Post review fixes of MWL#148 (moving max/min optimization in optimize phase).
sql/item_subselect.cc:
  Cleanup. Comments added.
sql/item_subselect.h:
  Cleanup.
sql/mysql_priv.h:
  Comments added.
sql/opt_subselect.cc:
  The function renamed and turned to method.
  Comments added.
sql/opt_subselect.h:
  The function turned to method of JOIN.
sql/sql_select.cc:
  Comment added. The function turned to method.
sql/sql_select.h:
  The function turned to method.
2011-05-12 00:14:15 +03:00
Sergey Petrunya
f34b421839 Code cleanup, added comments. 2011-05-10 11:31:02 +01:00
Sergey Petrunya
76fd8c1c2f Change "disable semi-joins in presense of outer joins" check to actually
check for outer joins (and not for any "t1 SOME_JOIN t2 ON cond" syntax).
2011-05-09 11:20:51 +01:00
Sergey Petrunya
524e6aad73 MWL#90: Non-merged semi-joins
- Take into account that grouping or aggregates decrease join output cardinality.
  (First code, can't make use of index statistics yet)
2011-05-09 10:35:55 +01:00
Oleksandr Byelkin
7b797fe66d Moving max/min optimization from prepare to optimization phase. MWL#148
mysql-test/r/explain.result:
  fixed results (new item)
mysql-test/r/subselect.result:
  fixed results (new item)
mysql-test/r/subselect_no_mat.result:
  fixed results (new item)
mysql-test/r/subselect_no_opts.result:
  fixed results (new item)
mysql-test/r/subselect_no_semijoin.result:
  Fixed results (new item)
mysql-test/suite/pbxt/r/subselect.result:
  Fixed results (new item)
mysql-test/t/explain.test:
  Fixed results (correct behaviour)
sql/item_cmpfunc.cc:
  Pass through  for max/min
sql/item_subselect.cc:
  moving max/min
sql/item_subselect.h:
  moving max/min
sql/mysql_priv.h:
  new uncacheble flags added
sql/opt_subselect.cc:
  maxmin moved.
sql/opt_subselect.h:
  New function for maxmin.
sql/sql_class.h:
  debug code
sql/sql_lex.cc:
  Fixed flags.
  Limit setting fixed.
sql/sql_lex.h:
  2 new flags.
sql/sql_select.cc:
  Prepare divided on 2 function to be able recollect some info after transformation.
sql/sql_select.h:
  Prepare divided on 2 functions.
2011-05-04 18:08:58 +03:00
unknown
5dc11616b2 MWL#89
Merge with 5.3
2011-05-02 21:59:16 +03:00
Sergey Petrunya
3098c21b8f Merge MWL#90 into 5.3-main 2011-04-30 04:59:05 +04:00
Igor Babaev
24edac2211 Merge 2011-04-26 21:11:06 -07:00
Igor Babaev
8d9dd21d85 Fixed LP bugs #717577, #724942.
Both these two bugs happened due to the following problem.
When a view column is referenced in the query an Item_direct_view_ref
object is created that is refers to the Item_field for the column.
All references to the same view column refer to the same Item_field.
Different references can belong to different AND/OR levels and,
as a result, can be included in different Item_equal object.
These Item_equal objects may include different constant objects.
If these constant objects are substituted for the Item_field created
for a view column we have a conflict situation when the second
substitution annuls the first substitution. This leads to
wrong result sets returned by the query. Bug #724942 demonstrates
such an erroneous behaviour.
Test case of the bug #717577 produces wrong result sets because best
equal fields of the multiple equalities built for different OR levels
of the WHERE condition differs. The subsitution for the best equal field
in the second OR branch overwrites the the substitution made for the
first branch.

To avoid such conflicts we have to substitute for the references
to the view columns rather than for the underlying field items.
To make such substitutions possible we have to include into
multiple equalities references to view columns rather than 
field items created for such columns.

This patch modifies the Item_equal class to include references
to view columns into multiple equality objects. It also performs
a clean up of the class methods and adds more comments. The methods
of the Item_direct_view_ref class that assist substitutions for
references to view columns has been also added by this patch.
2011-04-26 19:58:41 -07:00
Sergei Golubchik
0accbd0364 lots of post-merge changes 2011-04-25 17:22:25 +02:00
Sergey Petrunya
acc161d363 BUG#752992: Wrong results for a subquery with 'semijoin=on'
- Let advance_sj_state() save the value of JOIN::cur_dups_producing_tables
  in POSITION::prefix_dups_producing_tables, and restore_sj_state() restore
  it.
2011-04-08 02:12:03 +04:00
Sergey Petrunya
81c6f73ff2 MWL#90: Address review feedback part #19: More comments 2011-04-05 14:33:15 +04:00
Sergey Petrunya
151207134b MWL#90: Address review feedback part #18 2011-04-05 12:20:03 +04:00
Sergey Petrunya
8fb724281e Merge MWL#90 with main 5.3 tree 2011-04-02 14:09:00 +04:00
unknown
952556b345 MWL#89
Merge 5.3 with 5.3-mwl89.
2011-03-30 10:10:59 +03:00
Sergey Petrunya
dc937fb53b MWL#90: Address review feedback part #16 2011-03-29 14:33:59 +04:00
Sergey Petrunya
4e5199912b Rename JOIN::top_jtrange_tables to top_join_tab_count 2011-03-27 03:54:15 +04:00
Sergey Petrunya
5de770f317 MWL#90: Address review feedback part #14 2011-03-27 03:45:16 +04:00
Sergey Petrunya
432a058723 - Fix valgrind failure introduced in previous push 2011-03-25 12:14:02 +03:00
Sergey Petrunya
39fe368d68 MWL#90: Address review feedback part #7 2011-03-22 16:46:39 +03:00
Sergey Petrunya
b77e3dc9f4 MWL#90: Address review feedback part #5 2011-03-22 13:09:55 +03:00
Sergey Petrunya
809a805251 MWL#90: Address review feedback part #4 2011-03-22 00:39:27 +03:00
Michael Widenius
139a2b64bf Merge with 5.2 2011-03-09 15:47:59 +02:00
unknown
0737fb479f MWL#89
Added comment.
2011-03-02 08:10:38 +02:00
unknown
7895c35874 MWL#89
Merge MWL#89 with 5.3.
2011-03-01 14:16:28 +02:00
unknown
96efe1cab3 Fix for LP BUG#714808 and LP BUG#719280.
The patch also adjusts several instable test results
to order the result.

Analysis:

The function prev_record_reads() may skip (jump over)
some query plan nodes where record_count < 1. At the
same time, even though get_partial_join_cost() uses
all first N plan nodes after the last constant table,
it may produce a smaller record_count than
prev_record_reads(), because the record count for
some plan nodes may be < 1, and these nodes may not
participate in prev_record_reads.

Solution:
The current solution is to treat the result of
get_partial_join_cost() as the upper bound for the
total number of unique lookup keys.
2011-02-15 22:17:18 +02:00
Sergey Petrunya
cdccdea951 MWL#90: Merge (4) 2011-02-15 22:25:10 +03:00
Sergey Petrunya
e4325ff60b MWL#90: Subqueries: Inside-out execution for materialized non-sj subqueries
- Merge with 5.3 (3)
2011-02-15 20:29:57 +03:00
unknown
6a66bf3182 MWL#89
Fixed LP BUG#714808 Assertion `outer_lookup_keys <= outer_record_count'

Analysis:

The function best_access_path() computes the number or records as
follows:

...
      if (rec < MATCHING_ROWS_IN_OTHER_TABLE)
        rec= MATCHING_ROWS_IN_OTHER_TABLE; // Fix for small tables
...
              if (table->quick_keys.is_set(key))
                records= (double) table->quick_rows[key];
              else
              {
                /* quick_range couldn't use key! */
                records= (double) s->records/rec;
              }

Above MATCHING_ROWS_IN_OTHER_TABLE == 10, and s->records == 1,
thus we get an estimated 0.1 records. As a result JOIN::get_partial_join_cost()
for the outer query computes outer_record_count == 0.1 records, which is
meaningless in this context.

Solution:
Round row count estimates that are < 1 to 1.
2011-02-10 16:23:59 +02:00
unknown
648e604615 MWL#89
Adjusted test cases in accordance with the implementation.
2011-02-03 17:00:28 +02:00
unknown
fd6a079993 MWL#89
- fixed incorrect query plans that resulted from emptying the
  keyuse array with the access methods of the non-modified query
  plan.
- fixed compiler warning.
2011-01-24 23:38:39 +02:00
Igor Babaev
ec368ab9fa Merge 2011-01-21 22:48:28 -08:00
Igor Babaev
0f46efcba6 Post-review fixes for the patch that added the code allowing to use
hash join over equi-join conditions without supporting indexes.
2011-01-21 18:21:55 -08:00
Sergei Golubchik
418df96d51 use bulk insert and repair by sort for unique keys in
Aria and MyISAM in create_internal_tmp_table_from_heap()
(safe, as duplicates are impossible).
This gives a HUGE speed boost!

sql/opt_subselect.cc:
  Fixed problem with wrong recinfo in create_duplicate_weedout_tmp_tabl()
  Tagged the table with 'no_rows' so that when we create the table on disk,
  we only store the index data. This gave us a major speedup!
sql/sql_select.cc:
  create_tmp_table_from_heap() now uses bulk_insert + repair_by_sort
  when creating Aria/MyISAM tables from HEAP tables.
  This gives a HUGE speed boost!
storage/maria/ha_maria.cc:
  Extended bulk_insert() to recreate UNIQUE keys for
  internal temporary tables
storage/maria/ma_open.c:
  Initialize m_info->lock.type properly for temporarly tables
  (needed for start_bulk_insert())
storage/maria/ma_write.c:
  Don't check uniques that are disabled
storage/myisam/ha_myisam.cc:
  Extended bulk_insert() to recreate UNIQUE keys for
  internal temporary tables.
2011-01-14 12:03:41 +01:00
Sergei Golubchik
6554977fe5 Added ha_write_tmp_row() for slightly faster write_row for internal temp tables.
This will also enable us in the future to collect statistics for
writes to internal tmp tables.

sql/handler.h:
  Added ha_write_tmp_row()
sql/opt_subselect.cc:
  ha_write_row -> ha_write_tmp_row
sql/sql_class.h:
  Added ha_write_tmp_row()
sql/sql_select.cc:
  ha_write_row -> ha_write_tmp_row
2011-01-14 11:58:45 +01:00
Sergei Golubchik
563da78d57 Removed some old comments.
mysys/my_handler.c:
  Fixed typo
2011-01-14 11:31:09 +01:00
Sergey Petrunya
770548442f Bug#46692 Crash occurring on queries with nested FROM subqueries using materialization
- Backport testcases
- We have a different fix because we've fixed part of the problem as part of fix for LPBUG#602574.
2011-01-15 02:07:04 +03:00
Igor Babaev
af800fd92f The patch adds the code that allows to use equi-join conditions
for hash join in the cases when there are no suitable indexes
for these conditions.
2011-01-04 21:59:41 -08:00
unknown
2e42948ed3 MWL#89
- Post-review fixes. Intermediate commit to address review point 1.6.
- Fixed valgrind warnings
2010-12-23 17:33:00 +02:00
Sergey Petrunya
7b9bcaa5ec MWL#90: Post-merge fixes: add mixing typecasts to remove compiler warnings 2010-12-17 02:35:11 +03:00
unknown
0bee625fea MWL#89
Post-review fixes. Intermediate commit to address
review points 1.1, 1.2, 1.3, 1.4, 1.5, and 3.1, 3.2, 3.3.
2010-12-15 12:54:25 +02:00
Sergey Petrunya
1bf3964fbe MWL#90: Subqueries: Inside-out execution for materialized non-sj subqueries
- Merge into 5.3-main
2010-12-09 18:07:59 +03:00
unknown
be95cde859 Merge 5.3-mwl89 into 5.3. 2010-12-03 15:37:54 +02:00
Michael Widenius
52090a4434 Code cleanup to get fewer reallocs() during execution.
- Changed TABLE->alias to String to get fewer reallocs when alias are used.
- Preallocate some buffers

Changed some String->c_ptr() -> String->ptr() when \0 is not needed.
Fixed wrong usage of String->ptr() when we need a \0 terminated string.
Use my_strtod() instead of my_atof() to avoid having to add \0 to string.
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
zr 

sql/event_db_repository.cc:
  Update usage of TABLE->alias
sql/event_scheduler.cc:
  c_ptr() -> c_ptr_safe()
sql/events.cc:
  c_ptr() -> ptr() as \0 was not needed
sql/field.cc:
  Update usage of TABLE->alias
sql/field.h:
  Update usage of TABLE->alias
sql/ha_partition.cc:
  Update usage of TABLE->alias
sql/handler.cc:
  Update usage of TABLE->alias
  Fixed wrong usage of str.ptr()
sql/item.cc:
  Fixed error where code wrongly assumed string was \0 terminated.
sql/item_func.cc:
  c_ptr() -> c_ptr_safe()
  Update usage of TABLE->alias
sql/item_sum.h:
  Use my_strtod() instead of my_atof() to avoid having to add \0 to string
sql/lock.cc:
  Update usage of TABLE->alias
sql/log.cc:
  c_ptr() -> ptr() as \0 was not needed
sql/log_event.cc:
  c_ptr_quick() -> ptr() as \0 was not needed
sql/opt_range.cc:
  ptr() -> c_ptr() as \0 is needed
sql/opt_subselect.cc:
  Update usage of TABLE->alias
sql/opt_table_elimination.cc:
  Update usage of TABLE->alias
sql/set_var.cc:
  ptr() -> c_ptr() as \0 is needed
  c_ptr() -> c_ptr_safe()
sql/sp.cc:
  c_ptr() -> ptr() as \0 was not needed
sql/sp_rcontext.cc:
  
  Update usage of TABLE->alias
sql/sql_base.cc:
  Preallocate buffers
  Update usage of TABLE->alias
sql/sql_class.cc:
  Fix arguments to sprintf() to work even if string is not \0 terminated
sql/sql_insert.cc:
  Update usage of TABLE->alias
  c_ptr() -> ptr() as \0 was not needed
sql/sql_load.cc:
  Preallocate buffers
  Trivial optimizations
sql/sql_parse.cc:
  Trivial optimization
sql/sql_plugin.cc:
  c_ptr() -> ptr() as \0 was not needed
sql/sql_select.cc:
  Update usage of TABLE->alias
sql/sql_show.cc:
  Update usage of TABLE->alias
sql/sql_string.h:
  Added move() function to move allocated memory from one object to another.
sql/sql_table.cc:
  Update usage of TABLE->alias
  c_ptr() -> c_ptr_safe()
sql/sql_test.cc:
  ptr() -> c_ptr_safe()
sql/sql_trigger.cc:
  Update usage of TABLE->alias
  c_ptr() -> c_ptr_safe()
sql/sql_update.cc:
  Update usage of TABLE->alias
sql/sql_view.cc:
  ptr() -> c_ptr_safe()
sql/sql_yacc.yy:
  ptr() -> c_ptr()
sql/table.cc:
  
  Update usage of TABLE->alias
sql/table.h:
  Changed TABLE->alias to String to get fewer reallocs when alias are used.
storage/federatedx/ha_federatedx.cc:
  Use c_ptr_safe() to ensure strings are \0 terminated.
storage/maria/ha_maria.cc:
  Update usage of TABLE->alias
storage/myisam/ha_myisam.cc:
  Update usage of TABLE->alias
storage/xtradb/row/row0sel.c:
  Ensure that null bits in record are properly reset.
  (Old code didn't work as row_search_for_mysql() can be called twice while reading fields from one row.
2010-11-24 00:08:48 +02:00
unknown
bc7369b74b MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
Merge 5.3-mwl89 into 5.3 main.

There is one remaining test failure in this merge:
innodb_mysql_lock2. All other tests have been checked to
deliver the same results/explains as 5.3-mwl89, including
the few remaining wrong results.
2010-11-05 14:42:58 +02:00
Sergey Petrunya
5d1c9ce470 LPBUG#602574: RQG: sql_select.cc:5385: bool greedy_search... : Assertion `join->best_read
- Make optimize_wo_join_buffering() handle cases where position->records_read=0 (this
  happens for outer joins that have constant tables inside them). The number of
  0 is not correct (should be 1 because outer join will produce at least a NULL-complemented
  record) but for now we just make it work with incorrect number.
2010-11-02 21:13:53 +03:00
unknown
9f2bddbd80 Fixed LP BUG#652727 and LP BUG#643424.
The fixes for #643424 was part of the fix for #652727, that's why both
fixes are pushed together.

- The cause for #643424 was the improper use of get_partial_join_cost(),
  which assumed that the 'n_tables' parameter was the upper bound for
  query plan node indexes.
  Fixed by generalizing get_partial_join_cost() as a method that computes
  the cost of any partial join.

- The cause of #652727 was that JOIN::choose_subquery_plan() incorrectly
  deleted the contents of the old keyuse array in the cases when an injected
  plan would not provide more key accesses, and reoptimization was not actually
  performed.
2010-11-02 15:27:01 +02:00
unknown
f670b6d22f MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
Added missing logic to handle the case when subquery tables are optimized
away early during optimization.
2010-10-23 21:28:58 +03:00
unknown
e85a4cb6b5 MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
- Added more tests to the MWL#89 specific test, and made the test more modular.
- Updated test files.
- Fixed a memory leak.
- More comments.

mysql-test/r/subselect_mat.result:
  - Updated the test file to reflect the new optimizer switches related to
    materialized subquery execution.
  - Added one extra test to test all cases that expose BUG#40037 (this is an old bug from 5.x).
  - Updated the test result with correct results that expose BUG#40037.
mysql-test/t/subselect_mat.test:
  - Updated the test file to reflect the new optimizer switches related to
    materialized subquery execution.
  - Added one extra test to test all cases that expose BUG#40037 (this is an old bug from 5.x).
  - Updated the test result with correct results that expose BUG#40037.
sql/sql_select.cc:
  Fixed a memory leak reported by Valgrind.
2010-10-20 15:43:55 +03:00
Sergey Petrunya
6765cc3017 # No BUG#, a case brought from 5.2's innodb_mysql_lock.test
- Fix a crash in nested semi-join subquery processing
2010-10-18 12:55:26 +04:00
Sergey Petrunya
a285ce3e58 Post-merge fixes part 1 2010-10-10 17:38:17 +03:00
unknown
addd57828d MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
- Changed the default optimizer switches to provide 5.1/5.2 compatible behavior
- Added a regression test file to test consistently all cases covered by MWL#89
- Added/corrected/improved comments.
2010-10-09 17:48:05 +03:00
unknown
4a3f135a2e MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
Added/corrected/improved comments.
2010-10-05 16:00:31 +03:00
unknown
8ec5e13f1f MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
Phase 3: Implementation of re-optimization of subqueries with injected predicates
           and cost comparison between Materialization and IN->EXISTS strategies.

The commit contains the following known problems:
- The implementation of EXPLAIN has not been re-engineered to reflect the
  changes in subquery optimization. EXPLAIN for subqueries is called during
  the execute phase, which results in different code paths during JOIN::optimize
  and thus in differing EXPLAIN messages for constant/system tables.
- There are some valgrind warnings that need investigation
- Several EXPLAINs with minor differences need to be reconsidered after fixing
  the EXPLAIN problem above.

This patch also adds one extra optimizer_switch: 'in_to_exists' for complete
manual control of the subquery execution strategies.
2010-09-30 18:32:44 +03:00
Igor Babaev
b969df8bbd Merge of the mwl106 tree into the latest 5.3 tree.
Resolved conflicts. Adjusted some test results
2010-09-23 08:10:53 -07:00
unknown
50888477af MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
* Fixed a "crack" between semijoin analysis and materialization analysis
  where semijoin didn't set the correct strategy for the IN predicate.

* Cosmetic changes in the code/comments.
2010-09-17 13:17:27 +03:00
unknown
2c81150590 MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
Reset correctly the LIMIT and the uncacheable flag of a subquery when executing
it via materialization.
2010-09-16 17:06:58 +03:00
unknown
35f0faf4c7 MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
- Corrected a wrong result that was recorded by the MySQL fix for BUG#39069.

- Removed Item_func_isnull::cached_value and all the logic around this custom-made
  caching of the NULL result because MWL#89 optimizes subqueries before the outer
  query is being executed, and this cache cannot be made easily to work for all
  kinds of Items (specifically Item_sum_sum, but others too).
2010-09-16 16:49:20 +03:00
unknown
18ad3bdc2f MWL#89: Cost-based choice between Materialization and IN->EXISTS transformation
Fixes for multiple problems/bugs/test failures that resulted from moving
subquery optimization from the execution phase to the optimization phase.
2010-09-05 18:43:47 +03:00
psergey
946aef4a58 MWL#90: code cleanup
- Remove deadcode
- Improve comments 
- Do small several small TODOs
2010-07-10 20:51:12 +03:00
unknown
ceb5468fd8 Subquery cache (MWL#66) added.
libmysqld/Makefile.am:
  The new file added.
mysql-test/r/index_merge_myisam.result:
  subquery_cache optimization option added.
mysql-test/r/myisam_mrr.result:
  subquery_cache optimization option added.
mysql-test/r/subquery_cache.result:
  The subquery cache tests added.
mysql-test/r/subselect3.result:
  Subquery cache switched off to avoid changing read statistics.
mysql-test/r/subselect3_jcl6.result:
  Subquery cache switched off to avoid changing read statistics.
mysql-test/r/subselect_no_mat.result:
  subquery_cache optimization option added.
mysql-test/r/subselect_no_opts.result:
  subquery_cache optimization option added.
mysql-test/r/subselect_no_semijoin.result:
  subquery_cache optimization option added.
mysql-test/r/subselect_sj.result:
  subquery_cache optimization option added.
mysql-test/r/subselect_sj_jcl6.result:
  subquery_cache optimization option added.
mysql-test/t/subquery_cache.test:
  The subquery cache tests added.
mysql-test/t/subselect3.test:
  Subquery cache switched off to avoid changing read statistics.
sql/CMakeLists.txt:
  The new file added.
sql/Makefile.am:
  The new files added.
sql/item.cc:
  Expression cache item (Item_cache_wrapper) added.
  Item_ref and Item_field fixed for correct usage of result field and fast resolwing in SP.
sql/item.h:
  Expression cache item (Item_cache_wrapper) added.
  Item_ref and Item_field fixed for correct usage of result field and fast resolwing in SP.
sql/item_cmpfunc.cc:
  Subquery cache added.
sql/item_cmpfunc.h:
  Subquery cache added.
sql/item_subselect.cc:
  Subquery cache added.
sql/item_subselect.h:
  Subquery cache added.
sql/item_sum.cc:
  Registration of subquery parameters added.
sql/mysql_priv.h:
  subquery_cache optimization option added.
sql/mysqld.cc:
  subquery_cache optimization option added.
sql/opt_range.cc:
  Fix due to subquery cache.
sql/opt_subselect.cc:
  Parameters of the function cahnged.
sql/procedure.h:
  .h file guard added.
sql/sql_base.cc:
  Registration of subquery parameters added.
sql/sql_class.cc:
  Option to allow add indeces to temporary table.
sql/sql_class.h:
  Item iterators added.
  Option to allow add indeces to temporary table.
sql/sql_expression_cache.cc:
  Expression cache for caching subqueries added.
sql/sql_expression_cache.h:
  Expression cache for caching subqueries added.
sql/sql_lex.cc:
  Registration of subquery parameters added.
sql/sql_lex.h:
  Registration of subqueries and subquery parameters added.
sql/sql_select.cc:
  Subquery cache added.
sql/sql_select.h:
  Subquery cache added.
sql/sql_union.cc:
  A new parameter to the function added.
sql/sql_update.cc:
  A new parameter to the function added.
sql/table.cc:
  Procedures to manage temporarty tables index added.
sql/table.h:
  Procedures to manage temporarty tables index added.
storage/maria/ha_maria.cc:
  Fix of handler to allow destoy a table in case of error during the table creation.
storage/maria/ha_maria.h:
  .h file guard added.
storage/myisam/ha_myisam.cc:
  Fix of handler to allow destoy a table in case of error during the table creation.
2010-07-10 13:37:30 +03:00
Sergey Petrunya
e9bac8db82 MWL#90, code cleanup
- Remove garbage comments
- fix a bug in join_tab_cmp
2010-06-11 18:43:06 +04:00
Sergey Petrunya
9316a60b2c MWL#90: code cleanup: remove garbage comments, etc. 2010-06-10 23:38:31 +04:00
Sergey Petrunya
b33826e555 MWL#90, code cleanup
- Unify EXPLAIN printout for <subqueryN> tables with regular tables
- Update test results for <subqueryN> tables:
  s/unique_key/distinct_key/g
  s/1.0/100.0/ for "filtered" column
2010-06-09 18:43:52 +04:00
Sergey Petrunya
b8fb052126 MWL#90, Code cleanup:
- Fix test failures: do not use make_cond_after_sjm()'s result in case
  of merged SJ-Materialization lookup
- Update test results
2010-06-09 16:50:12 +04:00
Sergey Petrunya
26c3dc455b MWL#90: Subqueries: Inside-out execution for non-semijoin materialized subqueries that are AND-parts of the WHERE
- Address feedback
- Code cleanup (not finished)
2010-06-08 18:22:31 +04:00
Sergey Petrunya
49568aefe7 MWL#90: Code cleanup: Unification of merged and non-merged SJM nests processing
- Make join buffering code to take into account that JOIN_TABs are not a 
  linear array anymore.
2010-06-05 12:37:16 +04:00
Sergey Petrunya
dad93f2c82 MWL#90, code movearound to unify merged and non-merged semi-join materialization processing
- First code, needs cleanup.
2010-06-04 17:40:57 +04:00
Sergey Petrunya
0cc3724697 MWL#90: Subqueries: Inside-out execution for non-semijoin materialized subqueries that are AND-parts of the WHERE
- Change "SUBQUERY#n" to "<subquery{n}>" in EXPLAIN output. We need to it to be
  lowercase so that EXPLAIN results do not differ in case between systems with
  case-sensitive and case-insensitive filesystems.
- Remove garbage comments, add better comments.
2010-05-27 16:14:25 +04:00
Igor Babaev
709a0a1310 MWL#106: Backport optimizations for derived tables and views.
The main consolidated patch.
2010-05-26 13:18:18 -07:00
Sergey Petrunya
a15e342d0b MWL#90: Subqueries: Inside-out execution for non-semijoin materialized subqueries that are AND-parts of the WHERE
- Remove JOIN::all_tables as it is not useful.
- Better wording in comments
2010-05-25 17:13:19 +04:00
Sergey Petrunya
62bf7f6124 MWL#90: Subqueries: Inside-out execution for non-semijoin materialized subqueries that are AND-parts of the WHERE
- Code cleanu.
- Make MWL#90 code require @@optimizer_switch='semijoin=on'
- Update test results with the above
- Fork subselect_mat.test - we want to check both semi-join materialization, 
  which now has broader scope and non-semijoin materialization.
2010-05-25 10:32:15 +04:00
Sergey Petrunya
5a53be655c MWL#90: Subqueries: Inside-out execution for non-semijoin materialized subqueries that are AND-parts of the WHERE
- Merge into 5.3-subqueries
2010-05-24 15:24:20 +04:00
Sergey Petrunya
61864021ca Code cleanup, remove junk comments 2010-05-24 15:17:38 +04:00
Sergey Petrunya
3f595889d3 Subqueries: Inside-out execution for non-semijoin materialized subqueries that are AND-parts of the WHERE
- Code cleanup
- Query plan change is due to s/ha_rows JOIN_TAB::read_time/double JOIN_TAB::read_time/
2010-05-23 23:13:18 +04:00
Sergey Petrunya
b9688830ec Subquery optimizations:
- Better comments
- Use more appropriate return types for functions
- Provide handling where it was missing.
2010-05-10 19:28:19 +04:00
Sergey Petrunya
b2c57cedbf Code cleanup in subquery optimizations 2010-04-25 12:23:52 +04:00
unknown
bdb00d844a Fixed an error in the creation of REF access method for materialized
semi-join, where the the REF buffer format was mistaken to be in
record format instead of key format. The error was that the null
byte for all fields of the record was in the front of the buffer,
and not before each field data.
2010-04-06 22:56:20 +03:00
Sergey Petrunya
2775f80f7d MWL#90: Subqueries: Inside-out execution for non-semijoin materialized
subqueries that are AND-parts of the WHERE
- First code (needs cleanup).
2010-04-06 00:16:45 +04:00
unknown
f59d1dcf0a Merge in MWL#68: Subquery optimization: Efficient NOT IN execution with NULLs 2010-03-15 21:52:58 +02:00
Sergey Petrunya
47d0cf2929 Apply fix by Roy Lyseng:
Bug#48623: Multiple subqueries are optimized incorrectly
      
The function setup_semijoin_dups_elimination() has a major loop that
goes through every table in the JOIN object. Usually, there is a normal
"plus one" increment in the for loop that implements this, but each semijoin
nest is treated as one entity and there is another increment that skips past
the semijoin nest to the next table in the JOIN object. However, when
combining these two increments, the next joined table is skipped, and if that
happens to be the start of another semijoin nest, the correct processing
for that nest will not be carried out.

mysql-test/r/subselect_sj.result:
  Added test results for bug#48623
mysql-test/r/subselect_sj_jcl6.result:
  Added test results for bug#48623
mysql-test/t/subselect_sj.test:
  Added test case for bug#48623
sql/opt_subselect.cc:
  Omitted the "plus one" increment in the for loop, added "plus one"
  in the remaining switch case, fixed coding style issue in remaining
  increment operations.
2010-03-15 09:32:54 +03:00
Sergey Petrunya
5c295e521a Merge 2010-03-14 21:25:43 +03:00
Sergey Petrunya
77e66c6985 BUG#43768: Prepared query with nested subqueries core dumps on second execution
Fix two problems:
1. Let optimize_semijoin_nests() reset sj_nest->sjmat_info irrespectively  
   of value of optimizer_flag. We need this in case somebody has turned optimization
   off between reexecutions of the same statement.

2. Do not pull out constant tables out of semi-join nests. The problem is that pullout 
   operation is not undoable, and if a table is constant because it is 1/0-row table it 
   may cease to be constant on the next execution. Note that tables that are constant
   because of possible eq_ref(const) access will still be pulled out as they are
   considered functionally-dependent.
2010-03-14 20:54:12 +03:00
Sergey Petrunya
ea982e336d Apply fix by oystein.grovlen@sun.com 2010-03-12:
Bug#48213 Materialized subselect crashes if using GEOMETRY type

The problem occurred because during semi-join a materialized table
was created which contained a GEOMETRY column, which is a specialized
BLOB column.  This caused an segmentation fault because such tables will
have extra columns, and the semi-join code was not prepared for that.

The solution is to disable materialization when Blob/Geometry columns would 
need to be materialized.  Blob columns cannot be used for index look-up 
anyway, so it does not makes sense to use materialization.

This fix implies that it is detected earlier that subquery materialization
can not be used.  The result of that is that in->exist optimization may
be performed for such queries.  Hence, extended query plans for such
queries had to be updated.

mysql-test/r/subselect_mat.result:
  Update extended query plan for subqueries that cannot use materialization
  due to Blobs.
mysql-test/r/subselect_sj.result:
  Updated result file.
mysql-test/r/subselect_sj_jcl6.result:
  Update result file.
mysql-test/t/subselect_sj.test:
  Add test case for Bug#48213 that verifies that semi-join works when subquery select list contain Blob columns.  Also verify that materialization is not
  used.
sql/opt_subselect.cc:
  Disable materialization for semi-join/subqueries when the subquery select list
  contain Blob columns.
2010-03-14 00:11:06 +03:00
Sergey Petrunya
02e9fa62e7 BUG#45174: XOR in subqueries produces differing results in 5.1 and 5.4
BUG#50019: Wrong result for IN-subquery with materialization
- Fix equality substitution in presense of semi-join materialization, lookup and scan variants
  (started off from fix by Evgen Potemkin, then modified it to work in all cases)
2010-03-13 23:04:52 +03:00
unknown
3d2a74609a MWL#68 Subquery optimization: Efficient NOT IN execution with NULLs
This patch does three things:
- It adds the possibility to force the execution of top-level [NOT] IN
  subquery predicates via the IN=>EXISTS transformation. This is done by
  setting both optimizer switches partial_match_rowid_merge and
  partial_match_table_scan to "off".
- It adjusts all test cases where the complete optimizer_switch is
  selected because now we have two more switches.
- For those test cases where the plan changes because of the new available
  strategies, we switch off both partial match strategies in order to
  force the "old" IN=>EXISTS strategy. This is done because most of these
  test cases specifically test bugs in this strategy.

sql/opt_subselect.cc:
  Adds the possibility to force the execution of top-level [NOT] IN
  subquery predicates via the IN=>EXISTS transformation. This is done by
  setting both optimizer switches partial_match_rowid_merge and
  partial_match_table_scan to "off".
2010-03-11 23:43:31 +02:00
unknown
292d466749 MWL#68 Subquery optimization: Efficient NOT IN execution with NULLs
Automerge with 5.3-subqueries
2010-03-09 12:36:15 +02:00
Sergey Petrunya
c2924e155e BUG#49129: Wrong result with IN-subquery with join_cache_level=6 and firstmatch=off
- The problem was that DuplicateWeedout strategy setup code wasn't aware of the 
  fact that join buffering will be used and applied optimization that doesn't work
  together with join buffering. Fixed by making DuplicateWeedout setup code to have 
  a pessimistic check about whether there is a chance that join buffering will be 
  used.
- Make JOIN_CACHE_BKA::init() correctly process Copy_field elements that denote saving
  current rowids in the join buffer.

mysql-test/r/subselect_sj2.result:
  Update test results
mysql-test/r/subselect_sj2_jcl6.result:
  Update test results
mysql-test/r/subselect_sj_jcl6.result:
  Testcase
mysql-test/t/subselect_sj2.test:
  Update test results
mysql-test/t/subselect_sj_jcl6.test:
  Testcase
sql/opt_subselect.cc:
  - The problem was that DuplicateWeedout strategy setup code wasn't aware of the 
    fact that join buffering will be used and applied optimization that doesn't work
    together with join buffering. Fixed by making DuplicateWeedout setup code to have 
    a pessimistic check about whether there is a chance that join buffering will be 
    used.
sql/sql_join_cache.cc:
  Make JOIN_CACHE_BKA::init() correctly process Copy_field elements that denote saving current rowids in the join buffer.
sql/sql_select.cc:
  Added a question note
2010-03-07 18:41:45 +03:00
Sergey Petrunya
3752aa72c9 Prepare for OJ+SJ handling: Make replace_where_subcondition() not to assume
it's working on the WHERE clause.
2010-02-25 08:09:10 +00:00
unknown
5515bcba06 MWL#68 Subquery optimization: Efficient NOT IN execution with NULLs
This patch implements correct NULL semantics for materialized subquery execution.
The implementation has the following properties and main limitations:
- It passes all query result tests, but fails a number of EXPLAIN tests because of
  changed plans.
- The EXPLAIN output for partial matching is not decided yet.
- It works only when all necessary indexes fit into main memory. Notice that these
  are not the general B-tree/Hash indexes, but instead much more compact ones,
  therefore this limitation may not be a problem in many practical cases.
- It doesn't contain specialized tests.
- In several places the implementation uses methods that are modified copies of
  other similar methods. These cases need to be refactored to avoid code duplication.
- Add a test if the predicate is top-level just before deciding on partial matching.
  If it is top-level, use a more efficient exec method (index lookup).
- Add sorting of indexes according to their selectivity. The code is almost there.
- Needs more comments, and to sync existing ones with the implementation.

sql/item_cmpfunc.h:
  Expose the Arg_comparator of a comparison predicate. This makes it possible to
  directly get the comparison result {-1,0,1}, which is not possible through the
  val_XXX() methods which "fold" such results into a boolean.
sql/item_subselect.cc:
  The core of the implementation of MWL#68.
sql/item_subselect.h:
  The core of the implementation of MWL#68.
sql/opt_subselect.cc:
  Removed the limitation for materialized subquery execution that it is applicable only
  for top-level predicates.
sql/sql_class.cc:
  New class select_materialize_with_stats that collects data statistics about
  the data being inserted into the target table.
sql/sql_class.h:
  New class select_materialize_with_stats that collects data statistics about
  the data being inserted into the target table.
sql/sql_select.cc:
  - more complete initialization of the TABLE object of a temp table.
  - call setup_subquery_materialization at one more exit point.
2010-02-19 23:55:57 +02:00
Sergey Petrunya
d63959eed3 Subquery optimizations backport: fix test failures, update test results. 2010-02-18 00:59:41 +03:00
Sergey Petrunya
fdd19a6c63 Subquery optimizations backport:
- Update test results
- More comments
- Add Item_in_optimizer::transform() which was lost in backport
2010-02-17 13:05:27 +03:00
Sergey Petrunya
e0bb2f90a2 Subquery optimization backport:
- Factor out subquery code into sql/opt_subselect.{h,cc}
- Stop using the term "confluent" (was used due to misreading the dictionary)
2010-02-16 00:53:06 +03:00