WHERE conditions
check_group_min_max() checks if the loose index scan
optimization is applicable for a given WHERE condition, that is
if the MIN/MAX attribute participates only in range predicates
comparing the corresponding field with constants.
The problem was that it considered the whole predicate suitable
for the loose index scan optimization as soon as it encountered
a constant as a predicate argument. This is obviously wrong for
cases when a constant is the first argument of a predicate
which does not satisfy the above condition.
Fixed check_group_min_max() so that all arguments of the input
predicate are considered to decide if it passes the test, even
though a constant has already been encountered.
mysql-test/r/group_min_max.result:
Added a test case for bug #48472.
mysql-test/t/group_min_max.test:
Added a test case for bug #48472.
sql/opt_range.cc:
Fixed check_group_min_max() so that all arguments of the input
predicate are considered to decide if it passes the test, even
though a constant has already been encountered.
- Moved some code from innodb_plugin to xtradb, to ensure that all tests runs
- Did changes in pbxt and maria storage engines becasue of changes in thd->query
- Reverted wrong code in sql_table.cc for how ROW_FORMAT is used.
This is a re-commit of Monty's merge to eliminate an extra commit from
MySQL-5.1.42 that was accidentally included in the merge.
This is a merge of the MySQL 5.1.41 clone-off (clone-5.1.41-build). In
case there are any extra changes done before final MySQL 5.1.41
release, these will need to be merged later before MariaDB 5.1.41
release.
When a query was using a DATE or DATETIME value formatted
using any other separator characters beside hyphen '-', a
query with a greater-or-equal '>=' condition matching only
the greatest value in an indexed column, the result was
empty if index range scan was employed.
The range optimizer got a new feature between 5.1.38 and
5.1.39 that changes a greater-or-equal condition to a
greater-than if the value matching that in the query was not
present in the table. But the value comparison function
compared the dates as strings instead of dates.
The bug was fixed by splitting the function
get_date_from_str in two: One part that parses and does
error checking. This function is now visible outside the
module. The old get_date_from_str now calls the new
function.
mysql-test/r/range.result:
Bug#47925: Test result
mysql-test/t/range.test:
Bug#47925: Test case
sql/item.cc:
Bug#47925: Fix + some edit on the comments
sql/item.h:
Bug#47925: Changed function signature
sql/item_cmpfunc.cc:
Bug#47925: Split function in two
sql/item_cmpfunc.h:
Bug#47925: Declaration of new function
sql/opt_range.cc:
Bug#47925: Added THD to function call
sql/time.cc:
Bug#47925: Added microsecond comparison
itself when running the pbxt.group_min_max test leading to
wrong result sets and execution plans for many test cases.
When evaluating the index merge scans the code ignored the following:
if an index merge contains a range tree, such that none of its
SEL_ARG trees represents index ranges, then the index merge must be
discarded as not usable for any index merge retrieval.
that blocked building index merge plans for the queries with where
conditions of the form
(key1|2_p1=c AND range(key1_p2)) OR (key1|2_p1=c AND range(key2_p2)).
The problem was discovered by Sergey Petrunia when he reviewed the patch
for WL#24.
Added new test cases. One of them failed to produce an index merge plan
before the patch was applied.
sql/opt_range.cc:
WL#3352 code review fixes
Corrected indenting and added range_par parameter for easier reading/formatting.
sql/partition_element.h:
WL#3352, changed formatting of copyright header
sql/sql_partition.cc:
WL#3352, fixed indenting and changed MAX_VALUE to MAXVALUE
sql/sql_partition.h:
WL#3352, fixed copyright header and indenting of
function header
The original code comes, as far as I know, from Google (Mark Callaghan's team) with additional work from Percona, Ourdelta and Weldon Whipple.
This code provides the same functionallity, but with a lot of changes to make it faster and better fit the MariaDB infrastucture.
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_sent, Rows_read
Added new variable / startup option 'userstat' to control if user statistics should be enabled or not
Added my_getcputime(); Returns cpu time used by this thread.
New FLUSH commands:
- FLUSH SLOW QUERY LOG
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
New SHOW commands:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
New Information schemas:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Added support for all new flush commands to mysqladmin
Added handler::ha_... wrappers for all handler read calls to do statistics counting
- Changed all code to use new ha_... calls
- Count number of read rows, changed rows and rows read trough an index
Added counting of number of bytes sent to binary log (status variable Binlog_bytes_written)
Added counting of access denied errors (status variable Access_denied_erors)
Bugs fixed:
- Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
- CLOCK_GETTIME was not propely working on Linuxm
client/mysqladmin.cc:
Added support for all new flush commmands and some common combinations:
flush-slow-log
flush-table-statistics
flush-index-statistics
flush-user-statistics
flush-client-statistics
flush-all-status
flush-all-statistics
configure.in:
Added checking if clock_gettime needs the librt.
(Fixes Bug #37639 clock_gettime is never used/enabled in Linux/Unix)
include/my_sys.h:
Added my_getcputime()
include/mysql_com.h:
Added LIST_PROCESS_HOST_LEN & new REFRESH target defines
mysql-test/r/information_schema.result:
New information schema tables added
mysql-test/r/information_schema_all_engines.result:
New information schema tables added
mysql-test/r/information_schema_db.result:
New information schema tables added
mysql-test/r/log_slow.result:
Added testing that flosh slow query logs is accepted
mysql-test/r/status_user.result:
Basic testing of user, client, table and index statistics
mysql-test/t/log_slow.test:
Added testing that flosh slow query logs is accepted
mysql-test/t/status_user-master.opt:
Ensure that we get a fresh restart before running status_user.test
mysql-test/t/status_user.test:
Basic testing of user, client, table and index statistics
mysys/my_getsystime.c:
Added my_getcputime()
Returns cpu time used by this thread.
sql/authors.h:
Updated authors to have core and original MySQL developers first.
sql/event_data_objects.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/event_db_repository.cc:
Changed to use new ha_... calls
sql/filesort.cc:
Changed to use new ha_... calls
sql/ha_partition.cc:
Changed to use new ha_... calls
Fixed comment syntax
sql/handler.cc:
Changed to use new ha_... calls
Reset table statistics
Added code to update global table and index status
Added counting of rows changed
sql/handler.h:
Added table and index statistics variables
Added function reset_statistics()
Added handler::ha_... wrappers for all handler read calls to do statistics counting
Protected all normal read calls to ensure we use the new calls in the server.
Made ha_partition a friend class so that partition code can call the old read functions
sql/item_subselect.cc:
Changed to use new ha_... calls
sql/lex.h:
Added keywords for new information schema tables and flush commands
sql/log.cc:
Added flush_slow_log()
Added counting of number of bytes sent to binary log
Removed not needed test of thd (It's used before, so it's safe to use)
Added THD object to MYSQL_BIN_LOG::write_cache() to simplify statistics counting
sql/log.h:
Added new parameter to write_cache()
Added flush_slow_log() functions.
sql/log_event.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/log_event_old.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/mysql_priv.h:
Updated call to mysql_reset_thd_for_next_command()
Added new statistics functions and variables needed by these.
sql/mysqld.cc:
Added new statistics variables and structures to handle these
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_set, Rows_read
Added new option 'userstat' to control if user statistics should be enabled or not
sql/opt_range.cc:
Changed to use new ha_... calls
sql/opt_range.h:
Changed to use new ha_... calls
sql/opt_sum.cc:
Changed to use new ha_... calls
sql/records.cc:
Changed to use new ha_... calls
sql/set_var.cc:
Added variable 'userstat'
sql/sp.cc:
Changed to use new ha_... calls
sql/sql_acl.cc:
Changed to use new ha_... calls
Added counting of access_denied_errors
sql/sql_base.cc:
Added call to statistics functions
sql/sql_class.cc:
Added usage of org_status_var, to store status variables at start of command
Added functions THD::update_stats(), THD::update_all_stats()
Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
sql/sql_class.h:
Added new status variables to status_var
Moved variables that was not ulong in status_var last.
Added variables to THD for storing temporary values during statistics counting
sql/sql_connect.cc:
Variables and functions to calculate user and client statistics
Added counting of access_denied_errors and lost_connections
sql/sql_cursor.cc:
Changed to use new ha_... calls
sql/sql_handler.cc:
Changed to use new ha_... calls
sql/sql_help.cc:
Changed to use new ha_... calls
sql/sql_insert.cc:
Changed to use new ha_... calls
sql/sql_lex.h:
Added SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, SQLCOM_SHOW_CLIENT_STATS
sql/sql_parse.cc:
Added handling of:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
Added handling of new FLUSH commands:
- FLUSH SLOW QUERY LOGS
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
Added THD parameter to mysql_reset_thd_for_next_command()
Added initialization and calls to user statistics functions
Added increment of statistics variables empty_queries, rows_sent and access_denied_errors.
Added counting of cpu time per query
sql/sql_plugin.cc:
Changed to use new ha_... calls
sql/sql_prepare.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/sql_select.cc:
Changed to use new ha_... calls
Indentation changes
sql/sql_servers.cc:
Changed to use new ha_... calls
sql/sql_show.cc:
Added counting of access denied errors
Added function for new information schema tables:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Changed to use new ha_... calls
sql/sql_table.cc:
Changed to use new ha_... calls
sql/sql_udf.cc:
Changed to use new ha_... calls
sql/sql_update.cc:
Changed to use new ha_... calls
sql/sql_yacc.yy:
Add new show and flush commands
sql/structs.h:
Add name_length to KEY to avoid some strlen
Added cache_name to KEY for fast storage of keyvalue in cache
Added structs USER_STATS, TABLE_STATS, INDEX_STATS
Added function prototypes for statistics functions
sql/table.cc:
Store db+table+index name into keyinfo->cache_name
sql/table.h:
Added new information schema tables
sql/tztime.cc:
Changed to use new ha_... calls
Problem: involving a spatial index for "non-spatial" queries
(that don't containt MBRXXX() functions) may lead to failed assert.
Fix: don't use spatial indexes in such cases.
mysql-test/r/gis-rtree.result:
Fix for bug#48258: Assertion failed when using a spatial index
- test result.
mysql-test/t/gis-rtree.test:
Fix for bug#48258: Assertion failed when using a spatial index
- test case.
sql/opt_range.cc:
Fix for bug#48258: Assertion failed when using a spatial index
- allow only spatial functions (MBRXXX) for itMBR keyparts.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
mysql-test/r/partition_pruning.result:
Fixed a broken test case.
mysql-test/r/range.result:
Added a test case for bug #47123.
mysql-test/r/subselect.result:
Fixed a broken test cases.
mysql-test/t/range.test:
Added a test case for bug #47123.
sql/opt_range.cc:
Fixed get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
The problem was in incorrect handling of predicates involving
NULL as a constant value by the range optimizer.
For example, when creating a SEL_ARG node from a condition of
the form "field < const" (which would normally result in the
"NULL < field < const" SEL_ARG), the special case when "const"
is NULL was not taken into account, so "NULL < field < NULL"
was produced for the "field < NULL" condition.
As a result, SEL_ARG structures of this form could not be
further optimized which in turn could lead to incorrectly
constructed SEL_ARG trees. In particular, code assuming SEL_ARG
structures to always form a sequence of ordered disjoint
intervals could enter an infinite loop under some
circumstances.
Fixed by changing get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
mysql-test/r/range.result:
Added a test case for bug #47123.
mysql-test/t/range.test:
Added a test case for bug #47123.
sql/opt_range.cc:
Fixed get_mm_leaf() so that for any sargable
predicate except "<=>" involving NULL as a constant, "empty"
SEL_ARG is returned, since such a predicate is always false.
"index_merge: fair choice between index_merge union and range access"
mysql-test/include/world.inc:
A new include file to upload the world database.
mysql-test/include/world_schema.inc:
A new include file to create tables of the world database.
mysql-test/r/index_merge_myisam.result:
The results for test cases testing the optimizations added in WL#24 for MyISAM.
mysql-test/r/range_vs_index_merge.result:
The results for test cases testing the optimizations added in WL#24 for InnoDB.
mysql-test/t/range_vs_index_merge.test:
Test cases to test the optimizations added in WL#24 for MyISAM.
mysql-test/t/range_vs_index_merge_innodb.test:
Test cases to test the optimizations added in WL#24 for InnoDB.
sql/sql_list.h:
Fixed a bug that did not allow adding a non-empty list to an empty list.
covering index
When two range predicates were combined under an OR
predicate, the algorithm tried to merge overlapping ranges
into one. But the case when a range overlapped several other
ranges was not handled. This lead to
1) ranges overlapping, which gave repeated results and
2) a range that overlapped several other ranges was cut off.
Fixed by
1) Making sure that a range got an upper bound equal to the
next range with a greater minimum.
2) Removing a continue statement
mysql-test/r/group_min_max.result:
Bug#42846: Changed query plans
mysql-test/r/range.result:
Bug#42846: Test result.
mysql-test/t/range.test:
Bug#42846: Test case.
sql/opt_range.cc:
Bug#42846: The fix.
Part1: Previously, both endpoints from key2 were copied,
which is not safe. Since ranges are processed in ascending
order of minimum endpoints, it is safe to copy the minimum
endpoint from key2 but not the maximum. The maximum may only
be copied if there is no other range or the other range's
minimum is greater than key2's maximum.
can lead to bad memory access
Problem: Field_bit is the only field which returns INT_RESULT
and doesn't have unsigned flag. As it's not a descendant of the
Field_num, so using ((Field_num *) field_bit)->unsigned_flag may lead
to unpredictable results.
Fix: check the field type before casting.
mysql-test/r/type_bit.result:
Fix for bug #42803: Field_bit does not have unsigned_flag field,
can lead to bad memory access
- test result.
mysql-test/t/type_bit.test:
Fix for bug #42803: Field_bit does not have unsigned_flag field,
can lead to bad memory access
- test case.
sql/opt_range.cc:
Fix for bug #42803: Field_bit does not have unsigned_flag field,
can lead to bad memory access
- don't cast to (Field_num *) Field_bit, as it's not a Field_num
descendant and is always unsigned by nature.
* Finished Monty and Jani's merge
* Some InnoDB tests still fail (because it's old xtradb code run against
newer testsuite). They are expected to go after mergning with the latest
xtradb.
results in server crash
check_group_min_max_predicates() assumed the input condition
item to be one of COND_ITEM, SUBSELECT_ITEM, or FUNC_ITEM.
Since a condition of the form "field" is also a valid condition
equivalent to "field <> 0", using such a condition in a query
where the loose index scan was chosen resulted in a debug
assertion failure.
Fixed by handling conditions of the FIELD_ITEM type in
check_group_min_max_predicates().
mysql-test/r/group_min_max.result:
Added a test case for bug #46607.
mysql-test/t/group_min_max.test:
Added a test case for bug #46607.
sql/opt_range.cc:
Handle conditions of the FUNC_ITEM type in
check_group_mix_max_predicates().
with gcc 4.3.2
This patch fixes a number of GCC warnings about variables used
before initialized. A new macro UNINIT_VAR() is introduced for
use in the variable declaration, and LINT_INIT() usage will be
gradually deprecated. (A workaround is used for g++, pending a
patch for a g++ bug.)
GCC warnings for unused results (attribute warn_unused_result)
for a number of system calls (present at least in later
Ubuntus, where the usual void cast trick doesn't work) are
also fixed.
client/mysqlmanager-pwgen.c:
A fix for warn_unused_result, adding fallback to use of
srand()/rand() if /dev/random cannot be used. Also actually
adds calls to rand() in the second branch so that it actually
creates a random password.
sql/opt_range.cc:
Removed duplicate code (if statement must have been duplicated during earlier merge).
sql/sql_partition.cc:
After mergeing bug#46362 and bug#20577, the NULL partition was also searched
when col = const, fixed by checking if = or range.
Problem was that the partition containing NULL values
was pruned away, since '2001-01-01' < '2001-02-00' but
TO_DAYS('2001-02-00') is NULL.
Added the NULL partition for RANGE/LIST partitioning on TO_DAYS()
function to be scanned too.
Also fixed a bug that added ALLOW_INVALID_DATES to sql_mode
(SELECT * FROM t WHERE date_col < '1999-99-99' on a RANGE/LIST
partitioned table would add it).
mysql-test/include/partition_date_range.inc:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Added include file to decrease test code duplication
mysql-test/r/partition_pruning.result:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Added test results
mysql-test/r/partition_range.result:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Updated test result.
This fix adds the partition containing NULL values to
the list of partitions to be scanned.
mysql-test/t/partition_pruning.test:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Added test case
sql/item.h:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Added MONOTONIC_*INCREASE_NOT_NULL values to be used by TO_DAYS.
sql/item_timefunc.cc:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Calculate the number of days as return value even for invalid dates.
This is so that pruning can be used even for invalid dates.
sql/opt_range.cc:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Fixed a bug that added ALLOW_INVALID_DATES to sql_mode
(SELECT * FROM t WHERE date_col < '1999-99-99' on a RANGE/LIST
partitioned table would add it).
sql/partition_info.h:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Resetting ret_null_part when a single partition is to be used, this
to avoid adding the NULL partition.
sql/sql_partition.cc:
Bug#20577: Partitions: use of to_days() function leads to selection failures
Always include the NULL partition if RANGE or LIST.
Use the returned value for the function for pruning, even if
it is marked as NULL, so that even '2000-00-00' can be
used for pruning, even if TO_DAYS('2000-00-00') is NULL.
Changed == to >= in get_next_partition_id_list to avoid
crash if part_iter->part_nums is not correctly setup.
There were a problem since pruning uses the field
for comparison (while evaluate_join_record uses longlong),
resulting in pruning failures when comparing DATE to DATETIME.
Fix was to always comparing DATE vs DATETIME as DATETIME,
by adding ' 00:00:00' to the DATE string.
And adding optimization for comparing with 23:59:59, so that
DATETIME_col > '2001-02-03 23:59:59' ->
TO_DAYS(DATETIME_col) > TO_DAYS('2001-02-03 23:59:59') instead
of '>='.
mysql-test/r/partition_pruning.result:
Bug#46362: Endpoint should be set to false for TO_DAYS(DATE)
Updated result-file
mysql-test/t/partition_pruning.test:
Bug#46362: Endpoint should be set to false for TO_DAYS(DATE)
Added testcases.
sql-common/my_time.c:
Bug#46362: Endpoint should be set to false for TO_DAYS(DATE)
removed duplicate assignment.
sql/item.cc:
Bug#46362: Endpoint should be set to false for TO_DAYS(DATE)
Changed field_is_equal_to_item into field_cmp_to_item, to
better handling DATE vs DATETIME comparision.
sql/item.h:
Bug#46362: Endpoint should be set to false for TO_DAYS(DATE)
Updated comment
sql/item_timefunc.cc:
Bug#46362: Endpoint should be set to false for TO_DAYS(DATE)
Added optimization (pruning) of DATETIME where time-part is
23:59:59
sql/opt_range.cc:
Bug#46362: Endpoint should be set to false for TO_DAYS(DATE)
Using the new stored_field_cmp_to_item for better pruning.
The code was using a special global buffer for the value of IS NULL ranges.
This was not always long enough to be copied by a regular memcpy. As a
result read buffer overflows may occur.
Fixed by setting the null byte to 1 and setting the rest of the field disk image
to NULL with a bzero (instead of relying on the buffer and memcpy()).
