Phase 3: Implementation of re-optimization of subqueries with injected predicates
and cost comparison between Materialization and IN->EXISTS strategies.
The commit contains the following known problems:
- The implementation of EXPLAIN has not been re-engineered to reflect the
changes in subquery optimization. EXPLAIN for subqueries is called during
the execute phase, which results in different code paths during JOIN::optimize
and thus in differing EXPLAIN messages for constant/system tables.
- There are some valgrind warnings that need investigation
- Several EXPLAINs with minor differences need to be reconsidered after fixing
the EXPLAIN problem above.
This patch also adds one extra optimizer_switch: 'in_to_exists' for complete
manual control of the subquery execution strategies.
Step2 in the separation of the creation of IN->EXISTS equi-join conditions from
their injection. The goal of this separation is to make it possible that the
IN->EXISTS conditions can be used for cost estimation without actually modifying
the subquery.
This patch separates row_value_in_to_exists_transformer() into two methods:
- create_row_value_in_to_exists_cond(), and
- inject_row_value_in_to_exists_cond()
The patch performs minimal refactoring of the code so that it is easier to solve
problems resulting from the separation. There is a lot to be simplified in this
code, but this will be done separately.
Step1 in the separation of the creation of IN->EXISTS equi-join conditions from
their injection. The goal of this separation is to make it possible that the
IN->EXISTS conditions can be used for cost estimation without actually modifying
the subquery.
This patch separates single_value_in_to_exists_transformer() into two methods:
- create_single_value_in_to_exists_cond(), and
- inject_single_value_in_to_exists_cond()
The patch performs minimal refactoring of the code so that it is easier to solve
problems resulting from the separation. There is a lot to be simplified in this
code, but this will be done separately.
1. Changed the lazy optimization for subqueries that can be
materialized into bottom-up optimization during the optimization of
the main query.
The main change is implemented by the method
Item_in_subselect::setup_engine.
All other changes were required to correct problems resulting from
changing the order of optimization. Most of these problems followed
the same pattern - there are some shared structures between a
subquery and its parent query. Depending on which one is optimized
first (parent or child query), these shared strucutres may get
different values, thus resulting in an inconsistent query plan.
2. Changed the code-generation for subquery materialization to be
performed in runtime memory for each (re)execution, instead of in
statement memory (once per prepared statement).
- Item_in_subselect::setup_engine() no longer creates materialization
related objects in statement memory.
- Merged subselect_hash_sj_engine::init_permanent and
subselect_hash_sj_engine::init_runtime into
subselect_hash_sj_engine::init, which is called for each
(re)execution.
- Fixed deletion of the temp table accordingly.
mysql-test/r/subselect_mat.result:
Adjusted changed EXPLAIN because of earlier optimization of subqueries.
Fixed compiler warnings.
queues.h and queues.c are now based on the UNIREG code and thus made BSD.
Fix code to use new queue() interface. This mostly affects how you access elements in the queue.
If USE_NET_CLEAR is not set, don't clear connection from unexpected characters. This should give a speed up when doing a lot of fast queries.
Fixed some code in ma_ft_boolean_search.c that had not made it from myisam/ft_boolean_search.c
include/queues.h:
Use UNIREG code base (BSD)
Changed init_queue() to take all initialization arguments.
New interface to access elements in queue
include/thr_alarm.h:
Changed to use time_t instead of ulong (portability)
Added index_in_queue, to be able to remove random element from queue in O(1)
mysys/queues.c:
Use UNIREG code base (BSD)
init_queue() and reinit_queue() now takes more initialization arguments. (No need for init_queue_ex() anymore)
Now one can tell queue_insert() to store in the element a pointer to where element is in queue. This allows one to remove elements from queue in O(1) instead of O(N)
mysys/thr_alarm.c:
Use new option in queue() to allow fast removal of elements.
Do less inside LOCK_alarm mutex.
This should give a major speed up of thr_alarm usage when there is many threads
sql/create_options.cc:
Fixed wrong printf
sql/event_queue.cc:
Use new queue interface()
sql/filesort.cc:
Use new queue interface()
sql/ha_partition.cc:
Use new queue interface()
sql/ha_partition.h:
Fixed compiler warning
sql/item_cmpfunc.cc:
Fixed compiler warning
sql/item_subselect.cc:
Use new queue interface()
Removed not used variable
sql/net_serv.cc:
If USE_NET_CLEAR is not set, don't clear connection from unexpected characters.
This should give a speed up when doing a lot of fast queries at the disadvantage that if there is a bug in the client protocol the connection will be dropped instead of being unnoticed.
sql/opt_range.cc:
Use new queue interface()
Fixed compiler warnings
sql/uniques.cc:
Use new queue interface()
storage/maria/ma_ft_boolean_search.c:
Copy code from myisam/ft_boolean_search.c
Use new queue interface()
storage/maria/ma_ft_nlq_search.c:
Use new queue interface()
storage/maria/ma_sort.c:
Use new queue interface()
storage/maria/maria_pack.c:
Use new queue interface()
Use queue_fix() instead of own loop to fix queue.
storage/myisam/ft_boolean_search.c:
Use new queue interface()
storage/myisam/ft_nlq_search.c:
Use new queue interface()
storage/myisam/mi_test_all.sh:
Remove temporary file from last run
storage/myisam/myisampack.c:
Use new queue interface()
Use queue_fix() instead of own loop to fix queue.
storage/myisam/sort.c:
Use new queue interface()
storage/myisammrg/myrg_queue.c:
Use new queue interface()
storage/myisammrg/myrg_rnext.c:
Use new queue interface()
storage/myisammrg/myrg_rnext_same.c:
Use new queue interface()
storage/myisammrg/myrg_rprev.c:
Use new queue interface()
libmysqld/Makefile.am:
The new file added.
mysql-test/r/index_merge_myisam.result:
subquery_cache optimization option added.
mysql-test/r/myisam_mrr.result:
subquery_cache optimization option added.
mysql-test/r/subquery_cache.result:
The subquery cache tests added.
mysql-test/r/subselect3.result:
Subquery cache switched off to avoid changing read statistics.
mysql-test/r/subselect3_jcl6.result:
Subquery cache switched off to avoid changing read statistics.
mysql-test/r/subselect_no_mat.result:
subquery_cache optimization option added.
mysql-test/r/subselect_no_opts.result:
subquery_cache optimization option added.
mysql-test/r/subselect_no_semijoin.result:
subquery_cache optimization option added.
mysql-test/r/subselect_sj.result:
subquery_cache optimization option added.
mysql-test/r/subselect_sj_jcl6.result:
subquery_cache optimization option added.
mysql-test/t/subquery_cache.test:
The subquery cache tests added.
mysql-test/t/subselect3.test:
Subquery cache switched off to avoid changing read statistics.
sql/CMakeLists.txt:
The new file added.
sql/Makefile.am:
The new files added.
sql/item.cc:
Expression cache item (Item_cache_wrapper) added.
Item_ref and Item_field fixed for correct usage of result field and fast resolwing in SP.
sql/item.h:
Expression cache item (Item_cache_wrapper) added.
Item_ref and Item_field fixed for correct usage of result field and fast resolwing in SP.
sql/item_cmpfunc.cc:
Subquery cache added.
sql/item_cmpfunc.h:
Subquery cache added.
sql/item_subselect.cc:
Subquery cache added.
sql/item_subselect.h:
Subquery cache added.
sql/item_sum.cc:
Registration of subquery parameters added.
sql/mysql_priv.h:
subquery_cache optimization option added.
sql/mysqld.cc:
subquery_cache optimization option added.
sql/opt_range.cc:
Fix due to subquery cache.
sql/opt_subselect.cc:
Parameters of the function cahnged.
sql/procedure.h:
.h file guard added.
sql/sql_base.cc:
Registration of subquery parameters added.
sql/sql_class.cc:
Option to allow add indeces to temporary table.
sql/sql_class.h:
Item iterators added.
Option to allow add indeces to temporary table.
sql/sql_expression_cache.cc:
Expression cache for caching subqueries added.
sql/sql_expression_cache.h:
Expression cache for caching subqueries added.
sql/sql_lex.cc:
Registration of subquery parameters added.
sql/sql_lex.h:
Registration of subqueries and subquery parameters added.
sql/sql_select.cc:
Subquery cache added.
sql/sql_select.h:
Subquery cache added.
sql/sql_union.cc:
A new parameter to the function added.
sql/sql_update.cc:
A new parameter to the function added.
sql/table.cc:
Procedures to manage temporarty tables index added.
sql/table.h:
Procedures to manage temporarty tables index added.
storage/maria/ha_maria.cc:
Fix of handler to allow destoy a table in case of error during the table creation.
storage/maria/ha_maria.h:
.h file guard added.
storage/myisam/ha_myisam.cc:
Fix of handler to allow destoy a table in case of error during the table creation.
- Item_in_subselect::init_left_expr_cache() should not try to
guess whether the left expression is accessed "over the grouping operation"
(i.e. the subselect is evaluated after the grouping while the left_expr is
an Item_ref that wraps an expression from before the grouping). Instead,
let new_Cached_item not to try accessing item->real_item() when creating
left expr cache.
Fixed some bugs introduced in 5.1.47
Disabled some tests until we have merged with latest Xtradb
configure.in:
Added testing if valgrind/memcheck.h exists
storage/pbxt/src/ha_pbxt.cc:
LOCK_plugin is not anymore locked in init
We should disable const subselect item evaluation because
subselect transformation does not happen in view_prepare_mode
and thus val_...() methods can not be called.
mysql-test/r/ctype_ucs.result:
test case
mysql-test/r/view.result:
test case
mysql-test/t/ctype_ucs.test:
test case
mysql-test/t/view.test:
test case
sql/item.cc:
disabled const subselect item evaluation in
view prepare mode.
sql/item_subselect.cc:
added Item_subselect::safe_charset_converter which
prevents const item evaluation in view prepare mode.
sql/item_subselect.h:
added Item_subselect::safe_charset_converter which
prevents const item evaluation in view prepare mode.
- Add Item_in_subselect::get_identifier() that returns subquery's id
- Change select_describe() to produce output in new format
- Update test results (checked)
query:
- When subquery is located in ORDER BY, EXPLAIN will run as follows:
select_describe() will run JOIN::prepare()/optimize() for the subquery;
then at some point subselect_single_select_engine::prepare() will be called,
which will create another join and run join->prepare().
In mainline mysql this is not a problem because subquery's join will be
destroyed after the first call.
In MariaDB, it won't (table elimination needs to keep JOIN objects around
for longer in order to know which tables were eliminated when constructing
EXPLAIN EXTENDED warning).
Fix the problem of memory leak by calling select_lex->cleanup() in
subselect_single_select_engine::prepare().
Bug#46949: memory leak with failed alter table to create partitions based on extract()
Bug#51830: Incorrect partition pruning on range partition (regression)
Fixed valgrind failure in select_describe(), read of uninitialized
Item_subselect::eliminated.
PBXT test file updates to reflect changes done in MySQL.
mysql-test/suite/pbxt/r/partition_error.result:
Result file update following MySQL 5.1.44 changes.
mysql-test/suite/pbxt/r/partition_pruning.result:
Result file update following MySQL 5.1.44 changes.
mysql-test/suite/pbxt/t/partition_error.test:
Test file update following MySQL 5.1.44 changes.
sql/item_subselect.cc:
Fixed valgrind failure in select_describe(), read of uninitialized
Item_subselect::eliminated:
- it turns out we can call select_describe() without having fixed
subquery items for child subselects. These are not the kind of subqueries
that we could eliminate, so the fix is to ensure that
item_subselect->eliminated==FALSE even before fix_fields is called.
Also added code to reset item_subselect->eliminated back to FALSE in
Item::reset() call.
sql/item_subselect.h:
Fixed valgrind failure in select_describe(), read of uninitialized
Item_subselect::eliminated:
- it turns out we can call select_describe() without having fixed
subquery items for child subselects. These are not the kind of subqueries
that we could eliminate, so the fix is to ensure that
item_subselect->eliminated==FALSE even before fix_fields is called.
Also added code to reset item_subselect->eliminated back to FALSE in
Item::reset() call.
sql/sql_partition.cc:
Fix Bug#51830: Revert part of the patch for Bug#49742, which caused the regression.
sql/table.cc:
Fix Bug#46949: memory leak in failed ALTER TABLE with partitioning.
The problem was that killing a query during the optimization
phase of a subselect would lead to crashes. The root of the
problem is that the subselect execution engine ignores failures
(eg: killed) during the optimization phase (JOIN::optimize),
leading to a crash once the subquery is executed due to
partially initialized structures (in this case a join tab).
The optimal solution would be to cleanup certain optimizer
structures if the optimization phase fails, but currently
there is no infrastructure to properly to track and cleanup
the structures. To workaround the whole problem one somewhat
good solution is to avoid executing a subselect if the query
has been killed. Cutting short any problems caused by failures
during the optimization phase.
sql/item_subselect.cc:
Do not execute a subselect if the session or query has been killed.
* Implemented a second partial matching strategy via table scan.
This strategy is a fallback when there is no memory for rowid merging.
* Refactored the selection and creation of partial matching strategies,
so that the choice of strategy is encapsulated in a separate method
choose_partial_match_strategy().
* Refactored the representation of partial match strategies so that:
- each strategy is represented by a polymorphic class, and
- the base class for all partial match strategies contains common
execution code.
* Added an estimate of the memory needed for the rowid merge strategy,
and the system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
* Added two optimizer_switch system variables to control the choice of
partial match strategy:
"partial_match_rowid_merge", "partial_match_table_scan".
* Fixed multiple problems with deallocation of resources by the partial
match strategies.
sql/mysql_priv.h:
* Added two optimizer_switch system variables to control the choice of
partial match strategy:
"partial_match_rowid_merge", "partial_match_table_scan".
sql/mysqld.cc:
* Added two optimizer_switch system variables to control the choice of
partial match strategy:
"partial_match_rowid_merge", "partial_match_table_scan".
* Added a system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
sql/set_var.cc:
* Added a system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
sql/sql_class.h:
* Added a system variable "rowid_merge_buff_size" to control the maximum
memory to be used by the rowid merge algorithm.
support-files/build-tags:
Newer versions of BZR require the recursive flag in order to list all files.
This patch mainly adds sorting of all indexes for partial matching
according to their NULL selectivity. The patch also fixes a related bug
in subselect_rowid_merge_engine::test_null_row() where the wrong matched
indexes were skipped.
In addition the patch:
- adds few ::print() methods,
- renames few variables that had similar names but different purpose.
In Item_ref::fix_fields() do invoke mark_as_dependent() for outside
references in all cases (see email for more details)
sql/item.cc:
In Item_ref::fix_fields() do invoke mark_as_dependent() for outside references in all cases.
sql/item.h:
Change Field_enumerator to enumerate Item_field-s not Field-s.
sql/item_subselect.cc:
Change Field_enumerator to enumerate Item_field-s not Field-s.
sql/opt_table_elimination.cc:
Change Field_enumerator to enumerate Item_field-s not Field-s.
Don't go into branch that calls upper_refs.empty() more than once per
PREPARE or EXECUTE
* Avoid crashing when processing references to outside from subquery's HAVING
(will explain in more details in email)
sql/item.h:
* Avoid crashing when processing references to outside from subquery's HAVING
(will explain in more details in email)
sql/item_subselect.cc:
* Better self-recursion protection in Item_subselect::fix_fields.
Don't go into branch that calls upper_refs.empty() more than once per
PREPARE or EXECUTE
This patch implements correct NULL semantics for materialized subquery execution.
The implementation has the following properties and main limitations:
- It passes all query result tests, but fails a number of EXPLAIN tests because of
changed plans.
- The EXPLAIN output for partial matching is not decided yet.
- It works only when all necessary indexes fit into main memory. Notice that these
are not the general B-tree/Hash indexes, but instead much more compact ones,
therefore this limitation may not be a problem in many practical cases.
- It doesn't contain specialized tests.
- In several places the implementation uses methods that are modified copies of
other similar methods. These cases need to be refactored to avoid code duplication.
- Add a test if the predicate is top-level just before deciding on partial matching.
If it is top-level, use a more efficient exec method (index lookup).
- Add sorting of indexes according to their selectivity. The code is almost there.
- Needs more comments, and to sync existing ones with the implementation.
sql/item_cmpfunc.h:
Expose the Arg_comparator of a comparison predicate. This makes it possible to
directly get the comparison result {-1,0,1}, which is not possible through the
val_XXX() methods which "fold" such results into a boolean.
sql/item_subselect.cc:
The core of the implementation of MWL#68.
sql/item_subselect.h:
The core of the implementation of MWL#68.
sql/opt_subselect.cc:
Removed the limitation for materialized subquery execution that it is applicable only
for top-level predicates.
sql/sql_class.cc:
New class select_materialize_with_stats that collects data statistics about
the data being inserted into the target table.
sql/sql_class.h:
New class select_materialize_with_stats that collects data statistics about
the data being inserted into the target table.
sql/sql_select.cc:
- more complete initialization of the TABLE object of a temp table.
- call setup_subquery_materialization at one more exit point.
- Marked a couple of tests with --big
- Fixed xtradb/handler/ha_innodb.cc to call explain_filename()
storage/xtradb/handler/ha_innodb.cc:
Call explain_filename() to get proper names for partitioned tables
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
Bug#41756 "Strange error messages about locks from InnoDB".
In JT_EQ_REF (join_read_key()) access method,
don't try to unlock rows in the handler, unless certain that
a) they were locked
b) they are not used.
Unlocking of rows is done by the logic of the nested join loop,
and is unaware of the possible caching that the access method may
have. This could lead to double unlocking, when a row
was unlocked first after reading into the cache, and then
when taken from cache, as well as to unlocking of rows which
were actually used (but taken from cache).
Delegate part of the unlocking logic to the access method,
and in JT_EQ_REF count how many times a record was actually
used in the join. Unlock it only if it's usage count is 0.
Implemented review comments.
mysql-test/r/innodb_lock_wait_timeout_1.result:
Update results (Bug41756).
mysql-test/t/innodb_lock_wait_timeout_1.test:
Add a test case (Bug#41756).
sql/item_subselect.cc:
Complete struct READ_RECORD initialization with a new
member to unlock records.
sql/records.cc:
Extend READ_RECORD API with a method to unlock read records.
sql/sql_select.cc:
In JT_EQ_REF (join_read_key()) access method,
don't try to unlock rows in the handler, unless certain that
a) they were locked
b) they are not used.
sql/sql_select.h:
Add members to TABLE_REF to count TABLE_REF buffer usage count.
sql/structs.h:
Update declarations.
Bug#41756 "Strange error messages about locks from InnoDB".
In JT_EQ_REF (join_read_key()) access method,
don't try to unlock rows in the handler, unless certain that
a) they were locked
b) they are not used.
Unlocking of rows is done by the logic of the nested join loop,
and is unaware of the possible caching that the access method may
have. This could lead to double unlocking, when a row
was unlocked first after reading into the cache, and then
when taken from cache, as well as to unlocking of rows which
were actually used (but taken from cache).
Delegate part of the unlocking logic to the access method,
and in JT_EQ_REF count how many times a record was actually
used in the join. Unlock it only if it's usage count is 0.
Implemented review comments.
mysql-test/r/bug41756.result:
Add result file (Bug#41756)
mysql-test/t/bug41756-master.opt:
Use --innodb-locks-unsafe-for-binlog, as in 5.0 just
using read_committed isolation is not sufficient to
reproduce the bug.
mysql-test/t/bug41756.test:
Add a test file (Bug#41756)
sql/item_subselect.cc:
Complete struct READ_RECORD initialization with a new
member to unlock records.
sql/records.cc:
Extend READ_RECORD API with a method to unlock read records.
sql/sql_select.cc:
In JT_EQ_REF (join_read_key()) access method,
don't try to unlock rows in the handler, unless certain that
a) they were locked
b) they are not used.
sql/sql_select.h:
Add members to TABLE_REF to count TABLE_REF buffer usage count.
sql/structs.h:
Update declarations.
If an outer query is broken, a subquery might not even get set up.
EXPLAIN EXTENDED did not expect this and merrily tried to de-ref all
of the half-setup info.
We now catch this case and print as much as we have, as it doesn't cost us
anything (doesn't make regular execution slower).
mysql-test/r/explain.result:
Show that EXPLAIN EXTENDED with subquery and illegal out query doesn't crash.
Show also that SHOW WARNINGS will render an additional Note in the hope of
being, well, helpful.
mysql-test/t/explain.test:
If we have only half a query for EXPLAIN EXTENDED to print (i.e.,
incomplete subquery info as outer query is illegal), we should
provide the user with as much info as we easily can if they ask
for it. What we should not do is crash when they come asking for
help, that violates etiquette in some countries.
sql/item_subselect.cc:
If the sub-query's actually set up, print it. Otherwise, elide.
When values of different types are compared they're converted to a type that
allows correct comparison. This conversion is done for each comparison and
takes some time. When a constant is being compared it's possible to cache the
value after conversion to speedup comparison. In some cases (large dataset,
complex WHERE condition with many type conversions) query might be executed
7% faster.
A test case isn't provided because all changes are internal and isn't visible
outside.
The behavior of the Item_cache is changed to cache values on the first request
of cached value rather than at the moment of storing item to be cached.
A flag named value_cached is added to the Item_cache class. It's set to TRUE
when cache holds the value of the last stored item.
Function named cache_value() is added to the Item_cache class and derived classes.
This function actually caches the value of the saved item.
Item_cache_xxx::store functions now only store item to be cached and set
value_cached flag to FALSE.
Item_cache_xxx::val_xxx functions are changed to call cache_value function
prior to returning cached value if value_cached is FALSE.
The Arg_comparator::set_cmp_func function now calls cache_converted_constant
to cache constants if they need a type conversion.
The Item_cache::get_cache function is overloaded to allow setting of the
cache type.
The cache_converted_constant function is added to the Arg_comparator class.
It checks whether a value can and should be cached and if so caches it.
sql/item.cc:
Bug#34384: Slow down on constant conversion.
Function named cache_value() is added to the Item_cache class and derived classes.
This function actually caches the value of the saved item.
Item_cache_xxx::store functions now only store item to be cached and set
value_cached flag to FALSE.
Item_cache_xxx::val_xxx functions are changed to call cache_value function
prior to returning cached value if value_cached is FALSE.
The Item_cache::get_cache function is overloaded to allow setting of the
cache type.
sql/item.h:
Bug#34384: Slow down on constant conversion.
A flag named value_cached is added to the Item_cache class. It's set to TRUE
when we need to start caching values when the store method is called.
Function named cache_value() is added to the Item_cache class and derived classes.
sql/item_cmpfunc.cc:
Bug#34384: Slow down on constant conversion.
A helper function cache_converted_constant is added to the Arg_comparator class.
It checks whether a given item can and should be cached and caches it if so.
The Arg_comparator::set_cmp_func function now calls cache_converted_constant
to cache constants if they need a type conversion.
sql/item_cmpfunc.h:
Bug#34384: Slow down on constant conversion.
The cache_converted_constant function is added to the Arg_comparator class.
It checks whether a value can and should be cached and if so caches it.
sql/item_subselect.cc:
Bug#34384: Slow down on constant conversion.
Force immediate caching of subselect result.
sql/item_xmlfunc.cc:
Bug#34384: Slow down on constant conversion.
sql/sp_rcontext.cc:
Bug#34384: Slow down on constant conversion.
Force immediate caching of values of an SP CASE function.
The original code comes, as far as I know, from Google (Mark Callaghan's team) with additional work from Percona, Ourdelta and Weldon Whipple.
This code provides the same functionallity, but with a lot of changes to make it faster and better fit the MariaDB infrastucture.
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_sent, Rows_read
Added new variable / startup option 'userstat' to control if user statistics should be enabled or not
Added my_getcputime(); Returns cpu time used by this thread.
New FLUSH commands:
- FLUSH SLOW QUERY LOG
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
New SHOW commands:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
New Information schemas:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Added support for all new flush commands to mysqladmin
Added handler::ha_... wrappers for all handler read calls to do statistics counting
- Changed all code to use new ha_... calls
- Count number of read rows, changed rows and rows read trough an index
Added counting of number of bytes sent to binary log (status variable Binlog_bytes_written)
Added counting of access denied errors (status variable Access_denied_erors)
Bugs fixed:
- Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
- CLOCK_GETTIME was not propely working on Linuxm
client/mysqladmin.cc:
Added support for all new flush commmands and some common combinations:
flush-slow-log
flush-table-statistics
flush-index-statistics
flush-user-statistics
flush-client-statistics
flush-all-status
flush-all-statistics
configure.in:
Added checking if clock_gettime needs the librt.
(Fixes Bug #37639 clock_gettime is never used/enabled in Linux/Unix)
include/my_sys.h:
Added my_getcputime()
include/mysql_com.h:
Added LIST_PROCESS_HOST_LEN & new REFRESH target defines
mysql-test/r/information_schema.result:
New information schema tables added
mysql-test/r/information_schema_all_engines.result:
New information schema tables added
mysql-test/r/information_schema_db.result:
New information schema tables added
mysql-test/r/log_slow.result:
Added testing that flosh slow query logs is accepted
mysql-test/r/status_user.result:
Basic testing of user, client, table and index statistics
mysql-test/t/log_slow.test:
Added testing that flosh slow query logs is accepted
mysql-test/t/status_user-master.opt:
Ensure that we get a fresh restart before running status_user.test
mysql-test/t/status_user.test:
Basic testing of user, client, table and index statistics
mysys/my_getsystime.c:
Added my_getcputime()
Returns cpu time used by this thread.
sql/authors.h:
Updated authors to have core and original MySQL developers first.
sql/event_data_objects.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/event_db_repository.cc:
Changed to use new ha_... calls
sql/filesort.cc:
Changed to use new ha_... calls
sql/ha_partition.cc:
Changed to use new ha_... calls
Fixed comment syntax
sql/handler.cc:
Changed to use new ha_... calls
Reset table statistics
Added code to update global table and index status
Added counting of rows changed
sql/handler.h:
Added table and index statistics variables
Added function reset_statistics()
Added handler::ha_... wrappers for all handler read calls to do statistics counting
Protected all normal read calls to ensure we use the new calls in the server.
Made ha_partition a friend class so that partition code can call the old read functions
sql/item_subselect.cc:
Changed to use new ha_... calls
sql/lex.h:
Added keywords for new information schema tables and flush commands
sql/log.cc:
Added flush_slow_log()
Added counting of number of bytes sent to binary log
Removed not needed test of thd (It's used before, so it's safe to use)
Added THD object to MYSQL_BIN_LOG::write_cache() to simplify statistics counting
sql/log.h:
Added new parameter to write_cache()
Added flush_slow_log() functions.
sql/log_event.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/log_event_old.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/mysql_priv.h:
Updated call to mysql_reset_thd_for_next_command()
Added new statistics functions and variables needed by these.
sql/mysqld.cc:
Added new statistics variables and structures to handle these
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_set, Rows_read
Added new option 'userstat' to control if user statistics should be enabled or not
sql/opt_range.cc:
Changed to use new ha_... calls
sql/opt_range.h:
Changed to use new ha_... calls
sql/opt_sum.cc:
Changed to use new ha_... calls
sql/records.cc:
Changed to use new ha_... calls
sql/set_var.cc:
Added variable 'userstat'
sql/sp.cc:
Changed to use new ha_... calls
sql/sql_acl.cc:
Changed to use new ha_... calls
Added counting of access_denied_errors
sql/sql_base.cc:
Added call to statistics functions
sql/sql_class.cc:
Added usage of org_status_var, to store status variables at start of command
Added functions THD::update_stats(), THD::update_all_stats()
Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
sql/sql_class.h:
Added new status variables to status_var
Moved variables that was not ulong in status_var last.
Added variables to THD for storing temporary values during statistics counting
sql/sql_connect.cc:
Variables and functions to calculate user and client statistics
Added counting of access_denied_errors and lost_connections
sql/sql_cursor.cc:
Changed to use new ha_... calls
sql/sql_handler.cc:
Changed to use new ha_... calls
sql/sql_help.cc:
Changed to use new ha_... calls
sql/sql_insert.cc:
Changed to use new ha_... calls
sql/sql_lex.h:
Added SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, SQLCOM_SHOW_CLIENT_STATS
sql/sql_parse.cc:
Added handling of:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
Added handling of new FLUSH commands:
- FLUSH SLOW QUERY LOGS
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
Added THD parameter to mysql_reset_thd_for_next_command()
Added initialization and calls to user statistics functions
Added increment of statistics variables empty_queries, rows_sent and access_denied_errors.
Added counting of cpu time per query
sql/sql_plugin.cc:
Changed to use new ha_... calls
sql/sql_prepare.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/sql_select.cc:
Changed to use new ha_... calls
Indentation changes
sql/sql_servers.cc:
Changed to use new ha_... calls
sql/sql_show.cc:
Added counting of access denied errors
Added function for new information schema tables:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Changed to use new ha_... calls
sql/sql_table.cc:
Changed to use new ha_... calls
sql/sql_udf.cc:
Changed to use new ha_... calls
sql/sql_update.cc:
Changed to use new ha_... calls
sql/sql_yacc.yy:
Add new show and flush commands
sql/structs.h:
Add name_length to KEY to avoid some strlen
Added cache_name to KEY for fast storage of keyvalue in cache
Added structs USER_STATS, TABLE_STATS, INDEX_STATS
Added function prototypes for statistics functions
sql/table.cc:
Store db+table+index name into keyinfo->cache_name
sql/table.h:
Added new information schema tables
sql/tztime.cc:
Changed to use new ha_... calls
* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
on subquery inside a SP
Problem: repeated call of a SP containing an incorrect query with a
subselect may lead to failed ASSERT().
Fix: set proper sublelect's state in case of error occured during
subquery transformation.
mysql-test/r/sp.result:
Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0'
on subquery inside a SP
- test result.
mysql-test/t/sp.test:
Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0'
on subquery inside a SP
- test case.
sql/item_subselect.cc:
Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0'
on subquery inside a SP
- don't set Item_subselect::changed in the Item_subselect::fix_fields()
if an error occured during subquery transformation.
That prevents us of further processing incorrect subqueries after
Item_in_subselect::select_in_like_transformer().
