mirror of
https://github.com/MariaDB/server.git
synced 2025-01-16 03:52:35 +01:00
b66cdbd1ea
This makes it easier to compare different costs and also allows
the optimizer to optimizer different storage engines more reliably.
- Added tests/check_costs.pl, a tool to verify optimizer cost calculations.
- Most engine costs has been found with this program. All steps to
calculate the new costs are documented in Docs/optimizer_costs.txt
- User optimizer_cost variables are given in microseconds (as individual
costs can be very small). Internally they are stored in ms.
- Changed DISK_READ_COST (was DISK_SEEK_BASE_COST) from a hard disk cost
(9 ms) to common SSD cost (400MB/sec).
- Removed cost calculations for hard disks (rotation etc).
- Changed the following handler functions to return IO_AND_CPU_COST.
This makes it easy to apply different cost modifiers in ha_..time()
functions for io and cpu costs.
- scan_time()
- rnd_pos_time() & rnd_pos_call_time()
- keyread_time()
- Enhanched keyread_time() to calculate the full cost of reading of a set
of keys with a given number of ranges and optional number of blocks that
need to be accessed.
- Removed read_time() as keyread_time() + rnd_pos_time() can do the same
thing and more.
- Tuned cost for: heap, myisam, Aria, InnoDB, archive and MyRocks.
Used heap table costs for json_table. The rest are using default engine
costs.
- Added the following new optimizer variables:
- optimizer_disk_read_ratio
- optimizer_disk_read_cost
- optimizer_key_lookup_cost
- optimizer_row_lookup_cost
- optimizer_row_next_find_cost
- optimizer_scan_cost
- Moved all engine specific cost to OPTIMIZER_COSTS structure.
- Changed costs to use 'records_out' instead of 'records_read' when
recalculating costs.
- Split optimizer_costs.h to optimizer_costs.h and optimizer_defaults.h.
This allows one to change costs without having to compile a lot of
files.
- Updated costs for filter lookup.
- Use a better cost estimate in best_extension_by_limited_search()
for the sorting cost.
- Fixed previous issues with 'filtered' explain column as we are now
using 'records_out' (min rows seen for table) to calculate filtering.
This greatly simplifies the filtering code in
JOIN_TAB::save_explain_data().
This change caused a lot of queries to be optimized differently than
before, which exposed different issues in the optimizer that needs to
be fixed. These fixes are in the following commits. To not have to
change the same test case over and over again, the changes in the test
cases are done in a single commit after all the critical change sets
are done.
InnoDB changes:
- Updated InnoDB to not divide big range cost with 2.
- Added cost for InnoDB (innobase_update_optimizer_costs()).
- Don't mark clustered primary key with HA_KEYREAD_ONLY. This will
prevent that the optimizer is trying to use index-only scans on
the clustered key.
- Disabled ha_innobase::scan_time() and ha_innobase::read_time() and
ha_innobase::rnd_pos_time() as the default engine cost functions now
works good for InnoDB.
Other things:
- Added --show-query-costs (\Q) option to mysql.cc to show the query
cost after each query (good when working with query costs).
- Extended my_getopt with GET_ADJUSTED_VALUE which allows one to adjust
the value that user is given. This is used to change cost from
microseconds (user input) to milliseconds (what the server is
internally using).
- Added include/my_tracker.h ; Useful include file to quickly test
costs of a function.
- Use handler::set_table() in all places instead of 'table= arg'.
- Added SHOW_OPTIMIZER_COSTS to sys variables. These are input and
shown in microseconds for the user but stored as milliseconds.
This is to make the numbers easier to read for the user (less
pre-zeros). Implemented in 'Sys_var_optimizer_cost' class.
- In test_quick_select() do not use index scans if 'no_keyread' is set
for the table. This is what we do in other places of the server.
- Added THD parameter to Unique::get_use_cost() and
check_index_intersect_extension() and similar functions to be able
to provide costs to called functions.
- Changed 'records' to 'rows' in optimizer_trace.
- Write more information to optimizer_trace.
- Added INDEX_BLOCK_FILL_FACTOR_MUL (4) and INDEX_BLOCK_FILL_FACTOR_DIV (3)
to calculate usage space of keys in b-trees. (Before we used numeric
constants).
- Removed code that assumed that b-trees has similar costs as binary
trees. Replaced with engine calls that returns the cost.
- Added Bitmap::find_first_bit()
- Added timings to join_cache for ANALYZE table (patch by Sergei Petrunia).
- Added records_init and records_after_filter to POSITION to remember
more of what best_access_patch() calculates.
- table_after_join_selectivity() changed to recalculate 'records_out'
based on the new fields from best_access_patch()
Bug fixes:
- Some queries did not update last_query_cost (was 0). Fixed by moving
setting thd->...last_query_cost in JOIN::optimize().
- Write '0' as number of rows for const tables with a matching row.
Some internals:
- Engine cost are stored in OPTIMIZER_COSTS structure. When a
handlerton is created, we also created a new cost variable for the
handlerton. We also create a new variable if the user changes a
optimizer cost for a not yet loaded handlerton either with command
line arguments or with SET
@@global.engine.optimizer_cost_variable=xx.
- There are 3 global OPTIMIZER_COSTS variables:
default_optimizer_costs The default costs + changes from the
command line without an engine specifier.
heap_optimizer_costs Heap table costs, used for temporary tables
tmp_table_optimizer_costs The cost for the default on disk internal
temporary table (MyISAM or Aria)
- The engine cost for a table is stored in table_share. To speed up
accesses the handler has a pointer to this. The cost is copied
to the table on first access. If one wants to change the cost one
must first update the global engine cost and then do a FLUSH TABLES.
This was done to be able to access the costs for an open table
without any locks.
- When a handlerton is created, the cost are updated the following way:
See sql/keycaches.cc for details:
- Use 'default_optimizer_costs' as a base
- Call hton->update_optimizer_costs() to override with the engines
default costs.
- Override the costs that the user has specified for the engine.
- One handler open, copy the engine cost from handlerton to TABLE_SHARE.
- Call handler::update_optimizer_costs() to allow the engine to update
cost for this particular table.
- There are two costs stored in THD. These are copied to the handler
when the table is used in a query:
- optimizer_where_cost
- optimizer_scan_setup_cost
- Simply code in best_access_path() by storing all cost result in a
structure. (Idea/Suggestion by Igor)
263 lines
9.6 KiB
C
263 lines
9.6 KiB
C
/* Copyright (c) 2006, 2010, Oracle and/or its affiliates. All rights reserved.
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
it under the terms of the GNU General Public License as published by
|
|
the Free Software Foundation; version 2 of the License.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with this program; if not, write to the Free Software
|
|
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */
|
|
|
|
/**
|
|
@file
|
|
File containing constants that can be used throughout the server.
|
|
|
|
@note This file shall not contain or include any declarations of any kinds.
|
|
*/
|
|
|
|
#ifndef SQL_CONST_INCLUDED
|
|
#define SQL_CONST_INCLUDED
|
|
|
|
#include <mysql_version.h>
|
|
|
|
#define LIBLEN FN_REFLEN-FN_LEN /* Max l{ngd p} dev */
|
|
/* extra 4+4 bytes for slave tmp tables */
|
|
#define MAX_DBKEY_LENGTH (NAME_LEN*2+1+1+4+4)
|
|
#define MAX_ALIAS_NAME 256
|
|
#define MAX_FIELD_NAME (NAME_LEN+1) /* Max colum name length +1 */
|
|
#define MAX_SYS_VAR_LENGTH 32
|
|
#define MAX_KEY MAX_INDEXES /* Max used keys */
|
|
#define MAX_REF_PARTS 32 /* Max parts used as ref */
|
|
|
|
/*
|
|
Maximum length of the data part of an index lookup key.
|
|
|
|
The "data part" is defined as the value itself, not including the
|
|
NULL-indicator bytes or varchar length bytes ("the Extras"). We need this
|
|
value because there was a bug where length of the Extras were not counted.
|
|
|
|
You probably need MAX_KEY_LENGTH, not this constant.
|
|
*/
|
|
#define MAX_DATA_LENGTH_FOR_KEY 3072
|
|
#if SIZEOF_OFF_T > 4
|
|
#define MAX_REFLENGTH 8 /* Max length for record ref */
|
|
#else
|
|
#define MAX_REFLENGTH 4 /* Max length for record ref */
|
|
#endif
|
|
#define MAX_HOSTNAME (HOSTNAME_LENGTH + 1) /* len+1 in mysql.user */
|
|
#define MAX_CONNECTION_NAME NAME_LEN
|
|
|
|
#define MAX_MBWIDTH 3 /* Max multibyte sequence */
|
|
#define MAX_FILENAME_MBWIDTH 5
|
|
#define MAX_FIELD_CHARLENGTH 255
|
|
/*
|
|
In MAX_FIELD_VARCHARLENGTH we reserve extra bytes for the overhead:
|
|
- 2 bytes for the length
|
|
- 1 byte for NULL bits
|
|
to avoid the "Row size too large" error for these three corner definitions:
|
|
CREATE TABLE t1 (c VARBINARY(65533));
|
|
CREATE TABLE t1 (c VARBINARY(65534));
|
|
CREATE TABLE t1 (c VARBINARY(65535));
|
|
Like VARCHAR(65536), they will be converted to BLOB automatically
|
|
in non-strict mode.
|
|
*/
|
|
#define MAX_FIELD_VARCHARLENGTH (65535-2-1)
|
|
#define MAX_FIELD_BLOBLENGTH UINT_MAX32 /* cf field_blob::get_length() */
|
|
#define CONVERT_IF_BIGGER_TO_BLOB 512 /* Threshold *in characters* */
|
|
|
|
/* Max column width +1 */
|
|
#define MAX_FIELD_WIDTH (MAX_FIELD_CHARLENGTH*MAX_MBWIDTH+1)
|
|
|
|
#define MAX_BIT_FIELD_LENGTH 64 /* Max length in bits for bit fields */
|
|
|
|
#define MAX_DATE_WIDTH 10 /* YYYY-MM-DD */
|
|
#define MIN_TIME_WIDTH 10 /* -HHH:MM:SS */
|
|
#define MAX_TIME_WIDTH 16 /* -DDDDDD HH:MM:SS */
|
|
#define MAX_TIME_FULL_WIDTH 23 /* -DDDDDD HH:MM:SS.###### */
|
|
#define MAX_DATETIME_FULL_WIDTH 26 /* YYYY-MM-DD HH:MM:SS.###### */
|
|
#define MAX_DATETIME_WIDTH 19 /* YYYY-MM-DD HH:MM:SS */
|
|
#define MAX_DATETIME_COMPRESSED_WIDTH 14 /* YYYYMMDDHHMMSS */
|
|
#define MAX_DATETIME_PRECISION 6
|
|
|
|
#define MAX_TABLES (sizeof(table_map)*8-3) /* Max tables in join */
|
|
#define PARAM_TABLE_BIT (((table_map) 1) << (sizeof(table_map)*8-3))
|
|
#define OUTER_REF_TABLE_BIT (((table_map) 1) << (sizeof(table_map)*8-2))
|
|
#define RAND_TABLE_BIT (((table_map) 1) << (sizeof(table_map)*8-1))
|
|
#define PSEUDO_TABLE_BITS (PARAM_TABLE_BIT | OUTER_REF_TABLE_BIT | \
|
|
RAND_TABLE_BIT)
|
|
#define CONNECT_STRING_MAXLEN 65535 /* stored in 2 bytes in .frm */
|
|
#define MAX_FIELDS 4096 /* Limit in the .frm file */
|
|
#define MAX_PARTITIONS 8192
|
|
|
|
#define MAX_SELECT_NESTING (SELECT_NESTING_MAP_SIZE - 1)
|
|
|
|
#define MAX_SORT_MEMORY 2048*1024
|
|
#define MIN_SORT_MEMORY 1024
|
|
|
|
/* Some portable defines */
|
|
|
|
#define STRING_BUFFER_USUAL_SIZE 80
|
|
|
|
/* Memory allocated when parsing a statement / saving a statement */
|
|
#define MEM_ROOT_BLOCK_SIZE 8192
|
|
#define MEM_ROOT_PREALLOC 8192
|
|
#define TRANS_MEM_ROOT_BLOCK_SIZE 4096
|
|
#define TRANS_MEM_ROOT_PREALLOC 4096
|
|
|
|
#define DEFAULT_ERROR_COUNT 64
|
|
#define EXTRA_RECORDS 10 /* Extra records in sort */
|
|
#define SCROLL_EXTRA 5 /* Extra scroll-rows. */
|
|
#define FIELD_NAME_USED ((uint) 32768) /* Bit set if fieldname used */
|
|
#define FORM_NAME_USED ((uint) 16384) /* Bit set if formname used */
|
|
#define FIELD_NR_MASK 16383 /* To get fieldnumber */
|
|
#define FERR -1 /* Error from my_functions */
|
|
#define CREATE_MODE 0 /* Default mode on new files */
|
|
#define NAMES_SEP_CHAR 255 /* Char to sep. names */
|
|
|
|
/*
|
|
This is used when reading large blocks, sequential read.
|
|
We assume that reading this much will be roughly the same cost as 1
|
|
seek / fetching one row from the storage engine.
|
|
Cost of one read of DISK_CHUNK_SIZE is DISK_SEEK_BASE_COST (ms).
|
|
*/
|
|
#define DISK_CHUNK_SIZE (uint) (65536) /* Size of diskbuffer for tmpfiles */
|
|
|
|
#define FRM_VER_TRUE_VARCHAR (FRM_VER+4) /* 10 */
|
|
#define FRM_VER_EXPRESSSIONS (FRM_VER+5) /* 11 */
|
|
#define FRM_VER_CURRENT FRM_VER_EXPRESSSIONS
|
|
|
|
/***************************************************************************
|
|
Configuration parameters
|
|
****************************************************************************/
|
|
|
|
#define ACL_CACHE_SIZE 256
|
|
#define MAX_PASSWORD_LENGTH 32
|
|
#define HOST_CACHE_SIZE 128
|
|
#define MAX_ACCEPT_RETRY 10 // Test accept this many times
|
|
#define MAX_FIELDS_BEFORE_HASH 32
|
|
#define USER_VARS_HASH_SIZE 16
|
|
#define SEQUENCES_HASH_SIZE 16
|
|
#define TABLE_OPEN_CACHE_MIN 200
|
|
#define TABLE_OPEN_CACHE_DEFAULT 2000
|
|
#define TABLE_DEF_CACHE_DEFAULT 400
|
|
/**
|
|
We must have room for at least 400 table definitions in the table
|
|
cache, since otherwise there is no chance prepared
|
|
statements that use these many tables can work.
|
|
Prepared statements use table definition cache ids (table_map_id)
|
|
as table version identifiers. If the table definition
|
|
cache size is less than the number of tables used in a statement,
|
|
the contents of the table definition cache is guaranteed to rotate
|
|
between a prepare and execute. This leads to stable validation
|
|
errors. In future we shall use more stable version identifiers,
|
|
for now the only solution is to ensure that the table definition
|
|
cache can contain at least all tables of a given statement.
|
|
*/
|
|
#define TABLE_DEF_CACHE_MIN 400
|
|
|
|
/**
|
|
Maximum number of connections default value.
|
|
151 is larger than Apache's default max children,
|
|
to avoid "too many connections" error in a common setup.
|
|
*/
|
|
#define MAX_CONNECTIONS_DEFAULT 151
|
|
|
|
/*
|
|
Stack reservation.
|
|
Feel free to raise this by the smallest amount you can to get the
|
|
"execution_constants" test to pass.
|
|
*/
|
|
#define STACK_MIN_SIZE 16000 // Abort if less stack during eval.
|
|
|
|
#define STACK_MIN_SIZE_FOR_OPEN (1024*80)
|
|
#define STACK_BUFF_ALLOC 352 ///< For stack overrun checks
|
|
#ifndef MYSQLD_NET_RETRY_COUNT
|
|
#define MYSQLD_NET_RETRY_COUNT 10 ///< Abort read after this many int.
|
|
#endif
|
|
|
|
#define QUERY_ALLOC_BLOCK_SIZE 16384
|
|
#define QUERY_ALLOC_PREALLOC_SIZE 24576
|
|
#define TRANS_ALLOC_BLOCK_SIZE 8192
|
|
#define TRANS_ALLOC_PREALLOC_SIZE 4096
|
|
#define RANGE_ALLOC_BLOCK_SIZE 4096
|
|
#define ACL_ALLOC_BLOCK_SIZE 1024
|
|
#define UDF_ALLOC_BLOCK_SIZE 1024
|
|
#define TABLE_ALLOC_BLOCK_SIZE 1024
|
|
#define WARN_ALLOC_BLOCK_SIZE 2048
|
|
#define WARN_ALLOC_PREALLOC_SIZE 1024
|
|
/*
|
|
Note that if we are using 32K or less, then TCmalloc will use a local
|
|
heap without locks!
|
|
*/
|
|
#define SHOW_ALLOC_BLOCK_SIZE (32768-MALLOC_OVERHEAD)
|
|
|
|
/*
|
|
The following parameters is to decide when to use an extra cache to
|
|
optimise seeks when reading a big table in sorted order
|
|
*/
|
|
#define MIN_FILE_LENGTH_TO_USE_ROW_CACHE (10L*1024*1024)
|
|
#define MIN_ROWS_TO_USE_TABLE_CACHE 100
|
|
#define MIN_ROWS_TO_USE_BULK_INSERT 100
|
|
|
|
/*
|
|
The lower bound of accepted rows when using filter.
|
|
This is used to ensure that filters are not too agressive.
|
|
*/
|
|
#define MIN_ROWS_AFTER_FILTERING 1.0
|
|
|
|
/**
|
|
Number of rows in a reference table when refered through a not unique key.
|
|
This value is only used when we don't know anything about the key
|
|
distribution.
|
|
*/
|
|
#define MATCHING_ROWS_IN_OTHER_TABLE 10
|
|
|
|
#define MY_CHARSET_BIN_MB_MAXLEN 1
|
|
|
|
/** Don't pack string keys shorter than this (if PACK_KEYS=1 isn't used). */
|
|
#define KEY_DEFAULT_PACK_LENGTH 8
|
|
|
|
/** Characters shown for the command in 'show processlist'. */
|
|
#define PROCESS_LIST_WIDTH 100
|
|
/* Characters shown for the command in 'information_schema.processlist' */
|
|
#define PROCESS_LIST_INFO_WIDTH 65535
|
|
|
|
#define PRECISION_FOR_DOUBLE 53
|
|
#define PRECISION_FOR_FLOAT 24
|
|
|
|
/* -[digits].E+## */
|
|
#define MAX_FLOAT_STR_LENGTH (FLT_DIG + 6)
|
|
/* -[digits].E+### */
|
|
#define MAX_DOUBLE_STR_LENGTH (DBL_DIG + 7)
|
|
|
|
/*
|
|
Default time to wait before aborting a new client connection
|
|
that does not respond to "initial server greeting" timely
|
|
*/
|
|
#define CONNECT_TIMEOUT 10
|
|
/* Wait 5 minutes before removing thread from thread cache */
|
|
#define THREAD_CACHE_TIMEOUT 5*60
|
|
|
|
/* The following can also be changed from the command line */
|
|
#define DEFAULT_CONCURRENCY 10
|
|
#define DELAYED_LIMIT 100 /**< pause after xxx inserts */
|
|
#define DELAYED_QUEUE_SIZE 1000
|
|
#define DELAYED_WAIT_TIMEOUT (5*60) /**< Wait for delayed insert */
|
|
#define MAX_CONNECT_ERRORS 100 ///< errors before disabling host
|
|
|
|
#define LONG_TIMEOUT ((ulong) 3600L*24L*365L)
|
|
|
|
/**
|
|
Maximum length of time zone name that we support (Time zone name is
|
|
char(64) in db). mysqlbinlog needs it.
|
|
*/
|
|
#define MAX_TIME_ZONE_NAME_LENGTH (NAME_LEN + 1)
|
|
|
|
#define SP_PSI_STATEMENT_INFO_COUNT 19
|
|
|
|
#endif /* SQL_CONST_INCLUDED */
|