This patch mainly adds sorting of all indexes for partial matching
according to their NULL selectivity. The patch also fixes a related bug
in subselect_rowid_merge_engine::test_null_row() where the wrong matched
indexes were skipped.
In addition the patch:
- adds few ::print() methods,
- renames few variables that had similar names but different purpose.
In Item_ref::fix_fields() do invoke mark_as_dependent() for outside
references in all cases (see email for more details)
sql/item.cc:
In Item_ref::fix_fields() do invoke mark_as_dependent() for outside references in all cases.
sql/item.h:
Change Field_enumerator to enumerate Item_field-s not Field-s.
sql/item_subselect.cc:
Change Field_enumerator to enumerate Item_field-s not Field-s.
sql/opt_table_elimination.cc:
Change Field_enumerator to enumerate Item_field-s not Field-s.
Don't go into branch that calls upper_refs.empty() more than once per
PREPARE or EXECUTE
* Avoid crashing when processing references to outside from subquery's HAVING
(will explain in more details in email)
sql/item.h:
* Avoid crashing when processing references to outside from subquery's HAVING
(will explain in more details in email)
sql/item_subselect.cc:
* Better self-recursion protection in Item_subselect::fix_fields.
Don't go into branch that calls upper_refs.empty() more than once per
PREPARE or EXECUTE
This patch implements correct NULL semantics for materialized subquery execution.
The implementation has the following properties and main limitations:
- It passes all query result tests, but fails a number of EXPLAIN tests because of
changed plans.
- The EXPLAIN output for partial matching is not decided yet.
- It works only when all necessary indexes fit into main memory. Notice that these
are not the general B-tree/Hash indexes, but instead much more compact ones,
therefore this limitation may not be a problem in many practical cases.
- It doesn't contain specialized tests.
- In several places the implementation uses methods that are modified copies of
other similar methods. These cases need to be refactored to avoid code duplication.
- Add a test if the predicate is top-level just before deciding on partial matching.
If it is top-level, use a more efficient exec method (index lookup).
- Add sorting of indexes according to their selectivity. The code is almost there.
- Needs more comments, and to sync existing ones with the implementation.
sql/item_cmpfunc.h:
Expose the Arg_comparator of a comparison predicate. This makes it possible to
directly get the comparison result {-1,0,1}, which is not possible through the
val_XXX() methods which "fold" such results into a boolean.
sql/item_subselect.cc:
The core of the implementation of MWL#68.
sql/item_subselect.h:
The core of the implementation of MWL#68.
sql/opt_subselect.cc:
Removed the limitation for materialized subquery execution that it is applicable only
for top-level predicates.
sql/sql_class.cc:
New class select_materialize_with_stats that collects data statistics about
the data being inserted into the target table.
sql/sql_class.h:
New class select_materialize_with_stats that collects data statistics about
the data being inserted into the target table.
sql/sql_select.cc:
- more complete initialization of the TABLE object of a temp table.
- call setup_subquery_materialization at one more exit point.
XOR conditions are not optimized, and Item_cond_xor therefore
acts like type Func_item even though it inherits from Item_cond.
A subtle difference between Item_func and Item_cond is that
you can get the children Items from the former by calling
arguments(), and from the latter by calling argument_list().
However, since Item_cond_xor inherits from Item_cond,
arguments() did not return any Items.
The fact that Item_cond_xor::arguments() did not return it's
children items lead to a problem for make_cond_for_index();
the method accepted that XOR items on unindexed columns were
pushed using ICP. ICP evaluation of non-indexed columns
does not (and should not) work.
The fix for this bug is to make Item_cond_xor return it's
children items when the arguments() method is used. This makes
Item_cond_xor behave more like Item_func and in turn allows
make_cond_for_index() to discover any conflicting children
Items.
This is a temporary fix and should be removed when Item_cond_xor
is optimized.
- Do call update_used_tables() for new conditions obtained when adding
outer join's triggered conditions. Correct values of used_tables() are
now needed for condition pushdown.
- Update test results
mysql-test/suite/pbxt/r/join_outer.result:
DS-MRR backport:
- Update test results
sql/sql_select.cc:
DS-MRR backport: fix buildbot valgrind failures:
- Do call update_used_tables() for new conditions obtained when adding
outer join's triggered conditions. Correct values of used_tables() are
now needed for condition pushdown.
- Add opt_range_mrr.cc file into source repo
mysql-test/r/myisam_mrr.result:
Make testcase work for both debug and release
mysql-test/t/myisam_mrr.test:
Make testcase work for both debug and release
sql/Makefile.am:
- Add opt_range_mrr.cc file into source repo
- Make index condition pushdown be controlled by an @@optimizer_switch flag,
not by @@engine_condition_pushdown
- Make MRR buffer size be controlled by @@mrr_buffer_size, not
by @@read_rnd_buffer_size
- Move parts of code to separate files
- Code cleanup
- Add --sorted_result to some SELECTs in tests.
WL#2474 "Multi Range Read: Change the default MRR implementation to implement new MRR interface"
WL#2475 "Batched range read functions for MyISAM/InnoDb"
"Index condition pushdown for MyISAM/InnoDB"
- Adjust test results (checked)
- Code cleanup.
WL#2474 "Multi Range Read: Change the default MRR implementation to implement new MRR interface"
WL#2475 "Batched range read functions for MyISAM/InnoDb"
"Index condition pushdown for MyISAM/InnoDB"
- Fix valgrind failures
WL#2474 "Multi Range Read: Change the default MRR implementation to implement new MRR interface"
WL#2475 "Batched range read functions for MyISAM/InnoDb"
"Index condition pushdown for MyISAM/InnoDB"
Igor's fix from sp1r-igor@olga.mysql.com-20080330055902-07614:
There could be observed the following problems:
1. EXPLAIN did not mention pushdown conditions from on expressions in the
'extra' column. As a result if a query had no where conditions pushed
down to a table, but had on conditions pushed to this table the 'extra'
column in the EXPLAIN for the table missed 'using where'.
2. Conditions for ref access were not eliminated from on expressions
though such conditions were eliminated from the where condition.
sql/sql_show.cc:
Change decimal item to type holder, to be able to define the decimals and max_length for the DECIMAL
field from the schema specification instead of the value of the item.
sql/mysqld.cc:
Explicit type casting required by windows x64 compiler.
storage/xtradb/include/srv0srv.h:
Parameters should be declared as ulong.
storage/xtradb/srv/srv0srv.c:
Parameters should be declared as ulong.
This makes them suitable for tar archices right away and also are easier to copy
Don't disable federated storage engine by default.
Don't allow one to disable the Maria storage engine if it's used for temp tables
BUILD/SETUP.sh:
Compile by default MySQL clients with libmysqldclient.a (not .so)
This makes them suitable for tar archices right away and also are easier to copy
scripts/make_binary_distribution.sh:
Abort it clients are compiled with the shared libmysqlclient.so library
sql/mysqld.cc:
Don't call kill_mysql() if signal handler is not setup (causes a core dump).
This is only relevant when starting with --gdb.
sql/sql_plugin.cc:
Don't disable federated storage engine by default.
Don't allow one to disable the Maria storage engine if it's used for temp tables
- Added checking of return value for system(), freopen(), fgets() and chown()
- Ensure that calls that require a format strings gets a format string
- Other trivial things
Updated test suite results (especially for pbxt and embedded server)
Removed warning for "Invalid (old?) table or database name 'mysqld.1'" from pbxt tests
Speed up some pbxt tests by inserting begin ; commit; around "while loops with inserts"
Added mysqld startup option '--debug-flush'
Create maria_recovery.trace in data directory instead of current directory
client/mysql.cc:
Check return value from system()
client/mysql_upgrade.c:
Check return value from fgets()
client/mysqladmin.cc:
Check return value from fgets()
client/mysqlslap.c:
Check return value from system() (but ignore it, as it's not critical)
extra/yassl/src/crypto_wrapper.cpp:
Check return value from fgets() (but ignore it, as it's internal file)
extra/yassl/taocrypt/src/aes.cpp:
Added extra {} to remove compiler warning
extra/yassl/taocrypt/src/blowfish.cpp:
Added extra {} to remove compiler warning
extra/yassl/taocrypt/src/misc.cpp:
Ifdef not used code
include/mysys_err.h:
Added error message for failing chown()
mysql-test/mysql-test-run.pl:
Don't give warning for skipping ndbcluster (never enabled in MariaDB)
mysql-test/suite/funcs_1/r/is_columns_is_embedded.result:
Update with new information schema information
mysql-test/suite/funcs_1/r/is_tables_is_embedded.result:
New test
mysql-test/suite/funcs_1/r/is_tables_myisam_embedded.result:
Update test results (has not been tested for a long time)
mysql-test/suite/funcs_1/r/is_tables_mysql_embedded.result:
Update test results (has not been tested for a long time)
mysql-test/suite/funcs_1/t/is_tables_is.test:
Don't run with embedded server (as results differ)
I added a new test for embedded server
mysql-test/suite/funcs_1/t/is_tables_is_embedded.test:
New test
mysql-test/suite/pbxt/my.cnf:
Allow one to run pbxt tests without having to specify --mysqld=--default-storage-engine=pbxt
mysql-test/suite/pbxt/t/count_distinct3.test:
Speed up test by inserting begin; ... commit;
mysql-test/suite/pbxt/t/subselect.test:
Speed up test by inserting begin; ... commit;
mysys/errors.c:
Added error message for failing chown()
mysys/my_copy.c:
Added error message for failing chown()
mysys/my_redel.c:
Added error message for failing chown()
mysys/safemalloc.c:
Added cast to get rid of compiler warning
sql/ha_partition.cc:
Fixed wrong argument to sql_print_error() (it requires a format string)
sql/log.cc:
Test return value of freopen()
sql/mysqld.cc:
Test return value of freopen()
Added startup option '--debug-flush' to be used when one gets a core dump (easy to explain to people on IRC)
sql/rpl_rli.cc:
Fixed wrong argument to sql_print_error() (it requires a format string)
sql/set_var.cc:
Added {} to get rid of compiler warnings
sql/slave.cc:
Fixed wrong argument to mi->report() and sql_print...() (they require a format string)
sql/sql_cache.cc:
Fixed wrong argument to sql_printinformation() (it requires a format string)
sql/sql_parse.cc:
Test return value of fgets()
sql/sql_plugin.cc:
Fixed wrong argument to sql_print_error() (it requires a format string)
sql/sql_select.cc:
Use unique table name for internal temp tables instead of full path
(Simple speed & space optimization)
sql/udf_example.c:
Removed compiler warning about not used variable
storage/maria/ha_maria.cc:
Fixed wrong argument to sql_print_error() and ma_check_print_error() (they require a format string)
storage/maria/ma_recovery.c:
Create maria_recovery.trace in data directory instead of current directory
storage/maria/unittest/ma_test_loghandler-t.c:
Fixed wrong argument to ok(); Requires a format string
storage/pbxt/src/strutil_xt.cc:
Detect temporary tables by checking if that path for the table is in the mysql data directory.
The database for temporary tables is after this patch, from PBXT point of view, ""
This is needed to stop PBXT from calling filename_to_tablename() with the base directory as an argument, which caused
ERROR: Invalid (old?) table or database name 'mysqld.1'" in the log when running the test suite.
tests/mysql_client_test.c:
Fixed compiler warnings
unittest/mysys/base64-t.c:
Fixed wrong argument to diag() (it requires a format string)
Added a comment that the current 'print' of differing buffers doesn't print the right thing,
but didn't fix this as it's not important (unless we find a bug in the real code)
- Review fixes
client/Makefile.am:
- Make it build on Linux
client/mysqlbinlog.cc:
- Coding style fixes
- Better/more comments
- Use client/sql_string.*, not server's sql/sql_string.*.
- Don't declare a dummy TABLE_LIST structure in the client.
client/sql_string.h:
- Use client/sql_string.*, not server's sql/sql_string.*.
sql/log_event.cc:
= Fix coding style
= Introduce Log_event::event_owns_temp_buf which tells whether Log_event::temp_buf is 'owned' by the Log_event object and should be my_free'd on return.
This is needed because rewrite_db() needs to dispose of the buffer, and
- when mysqlbinlog is reading directly from binlog file, the buffer
should be freed
- when mysqlbinlog is reading from a server, the buffer is a part of network
buffer and shouldn't be freed.
sql/log_event.h:
Introduce Log_event::event_owns_temp_buf which tells whether Log_event::temp_buf is 'owned' by the Log_event object and should be my_free'd on return.
This is needed because rewrite_db() needs to dispose of the buffer, and
- when mysqlbinlog is reading directly from binlog file, the buffer
should be freed
- when mysqlbinlog is reading from a server, the buffer is a part of network
buffer and shouldn't be freed.
sql/mysqld.cc:
- Better/more comments
sql/rpl_filter.cc:
- #ifdef-out Rpl_filter::tables_ok from the client. This allows not
to define dummy TABLE_LIST on the client
sql/rpl_filter.h:
- #ifdef-out Rpl_filter::tables_ok from the client. This allows not
to define dummy TABLE_LIST on the client
sql/sql_string.cc:
- Use client/sql_string.*, not server's sql/sql_string.*.
sql/sql_string.h:
- Use client/sql_string.*, not server's sql/sql_string.*.
- mysql_install_db doesn't require --basedir option anymore
- Fix that mysql_install_db mysqld_safe mysql_fix_privilege_tables has right paths for tar binary install
- Move some files from share/mysql to where they have always been (1.0 - 5.0)
- Move scripts used when installing MariaDB to the scripts directory (Added symlinks in old place for compatibility)
- Move man files to 'man' (from 'share/man')
- Ensure that all scripts are executable
mysql_secure_installation.sh
- Added support for --basedir flag
- Don't assume that the mysql binary is in the path
Other:
- Fixed crash when one got kill signal during bootstrap
Docs/INSTALL-BINARY:
Change references from MySQL to MariaDB
Docs/mysql.info:
Change references from MySQL to MariaDB
README:
Removed duplicate row
configure.in:
Change references from MySQL to MariaDB
libmysql/libmysql.c:
Assume that if version string contains '-maria-' we are also using MariaDB
scripts/make_binary_distribution.sh:
Fix that mysql_install_db mysqld_safe mysql_fix_privilege_tables has right paths for tar binary install
Remove some files that should not be part of a MariaDB binary install
Move some files from share/mysql to where they have always been (1.0 - 5.0)
Move man files to 'man' (from 'share/man')
Move scripts used when installing MariaDB to the scripts directory (Added symlinks in old place for compatibility)
Removed generation of 'netware/init_db.sql', as it was never used
Remove old reference to bk tree
scripts/mysql_secure_installation.sh:
Added support for --basedir flag
Don't assume that the mysql binary is in the path
Use .my.cnf files
Removed a lot of useless 'echo' rows
sql/mysqld.cc:
Fixed crash when one got kill signal during bootstrap
storage/maria/Makefile.am:
Don't add internal test files to binary distributions
storage/myisam/Makefile.am:
Don't add internal test files to binary distributions
support-files/Makefile.am:
Make scripts executable
support-files/config.huge.ini.sh:
Removed execution bit
support-files/config.medium.ini.sh:
Removed execution bit
support-files/config.small.ini.sh:
Removed execution bit
support-files/mysql.server.sh:
Ensure this works also mysqld is in the 'bin' directory
The original code comes, as far as I know, from Google (Mark Callaghan's team) with additional work from Percona, Ourdelta and Weldon Whipple.
This code provides the same functionallity, but with a lot of changes to make it faster and better fit the MariaDB infrastucture.
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_sent, Rows_read
Added new variable / startup option 'userstat' to control if user statistics should be enabled or not
Added my_getcputime(); Returns cpu time used by this thread.
New FLUSH commands:
- FLUSH SLOW QUERY LOG
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
New SHOW commands:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
New Information schemas:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Added support for all new flush commands to mysqladmin
Added handler::ha_... wrappers for all handler read calls to do statistics counting
- Changed all code to use new ha_... calls
- Count number of read rows, changed rows and rows read trough an index
Added counting of number of bytes sent to binary log (status variable Binlog_bytes_written)
Added counting of access denied errors (status variable Access_denied_erors)
Bugs fixed:
- Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
- CLOCK_GETTIME was not propely working on Linuxm
client/mysqladmin.cc:
Added support for all new flush commmands and some common combinations:
flush-slow-log
flush-table-statistics
flush-index-statistics
flush-user-statistics
flush-client-statistics
flush-all-status
flush-all-statistics
configure.in:
Added checking if clock_gettime needs the librt.
(Fixes Bug #37639 clock_gettime is never used/enabled in Linux/Unix)
include/my_sys.h:
Added my_getcputime()
include/mysql_com.h:
Added LIST_PROCESS_HOST_LEN & new REFRESH target defines
mysql-test/r/information_schema.result:
New information schema tables added
mysql-test/r/information_schema_all_engines.result:
New information schema tables added
mysql-test/r/information_schema_db.result:
New information schema tables added
mysql-test/r/log_slow.result:
Added testing that flosh slow query logs is accepted
mysql-test/r/status_user.result:
Basic testing of user, client, table and index statistics
mysql-test/t/log_slow.test:
Added testing that flosh slow query logs is accepted
mysql-test/t/status_user-master.opt:
Ensure that we get a fresh restart before running status_user.test
mysql-test/t/status_user.test:
Basic testing of user, client, table and index statistics
mysys/my_getsystime.c:
Added my_getcputime()
Returns cpu time used by this thread.
sql/authors.h:
Updated authors to have core and original MySQL developers first.
sql/event_data_objects.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/event_db_repository.cc:
Changed to use new ha_... calls
sql/filesort.cc:
Changed to use new ha_... calls
sql/ha_partition.cc:
Changed to use new ha_... calls
Fixed comment syntax
sql/handler.cc:
Changed to use new ha_... calls
Reset table statistics
Added code to update global table and index status
Added counting of rows changed
sql/handler.h:
Added table and index statistics variables
Added function reset_statistics()
Added handler::ha_... wrappers for all handler read calls to do statistics counting
Protected all normal read calls to ensure we use the new calls in the server.
Made ha_partition a friend class so that partition code can call the old read functions
sql/item_subselect.cc:
Changed to use new ha_... calls
sql/lex.h:
Added keywords for new information schema tables and flush commands
sql/log.cc:
Added flush_slow_log()
Added counting of number of bytes sent to binary log
Removed not needed test of thd (It's used before, so it's safe to use)
Added THD object to MYSQL_BIN_LOG::write_cache() to simplify statistics counting
sql/log.h:
Added new parameter to write_cache()
Added flush_slow_log() functions.
sql/log_event.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/log_event_old.cc:
Updated call to mysql_reset_thd_for_next_command()
Changed to use new ha_... calls
sql/mysql_priv.h:
Updated call to mysql_reset_thd_for_next_command()
Added new statistics functions and variables needed by these.
sql/mysqld.cc:
Added new statistics variables and structures to handle these
Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_set, Rows_read
Added new option 'userstat' to control if user statistics should be enabled or not
sql/opt_range.cc:
Changed to use new ha_... calls
sql/opt_range.h:
Changed to use new ha_... calls
sql/opt_sum.cc:
Changed to use new ha_... calls
sql/records.cc:
Changed to use new ha_... calls
sql/set_var.cc:
Added variable 'userstat'
sql/sp.cc:
Changed to use new ha_... calls
sql/sql_acl.cc:
Changed to use new ha_... calls
Added counting of access_denied_errors
sql/sql_base.cc:
Added call to statistics functions
sql/sql_class.cc:
Added usage of org_status_var, to store status variables at start of command
Added functions THD::update_stats(), THD::update_all_stats()
Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
sql/sql_class.h:
Added new status variables to status_var
Moved variables that was not ulong in status_var last.
Added variables to THD for storing temporary values during statistics counting
sql/sql_connect.cc:
Variables and functions to calculate user and client statistics
Added counting of access_denied_errors and lost_connections
sql/sql_cursor.cc:
Changed to use new ha_... calls
sql/sql_handler.cc:
Changed to use new ha_... calls
sql/sql_help.cc:
Changed to use new ha_... calls
sql/sql_insert.cc:
Changed to use new ha_... calls
sql/sql_lex.h:
Added SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, SQLCOM_SHOW_CLIENT_STATS
sql/sql_parse.cc:
Added handling of:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS
Added handling of new FLUSH commands:
- FLUSH SLOW QUERY LOGS
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS
Added THD parameter to mysql_reset_thd_for_next_command()
Added initialization and calls to user statistics functions
Added increment of statistics variables empty_queries, rows_sent and access_denied_errors.
Added counting of cpu time per query
sql/sql_plugin.cc:
Changed to use new ha_... calls
sql/sql_prepare.cc:
Updated call to mysql_reset_thd_for_next_command()
sql/sql_select.cc:
Changed to use new ha_... calls
Indentation changes
sql/sql_servers.cc:
Changed to use new ha_... calls
sql/sql_show.cc:
Added counting of access denied errors
Added function for new information schema tables:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS
Changed to use new ha_... calls
sql/sql_table.cc:
Changed to use new ha_... calls
sql/sql_udf.cc:
Changed to use new ha_... calls
sql/sql_update.cc:
Changed to use new ha_... calls
sql/sql_yacc.yy:
Add new show and flush commands
sql/structs.h:
Add name_length to KEY to avoid some strlen
Added cache_name to KEY for fast storage of keyvalue in cache
Added structs USER_STATS, TABLE_STATS, INDEX_STATS
Added function prototypes for statistics functions
sql/table.cc:
Store db+table+index name into keyinfo->cache_name
sql/table.h:
Added new information schema tables
sql/tztime.cc:
Changed to use new ha_... calls
