mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-18 13:02:28 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/sql/field.cc

9357 lines
251 KiB
C++
Raw Normal View History

Import changeset
2000-07-31 21:29:14 +02:00
/* Copyright (C) 2000 MySQL AB & MySQL Finland AB & TCX DataKonsult AB
Update copyright Fixed memory leak on shutdown (Affects the embedded version & MyODBC)
2001-12-06 13:10:51 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
Update copyright Fixed memory leak on shutdown (Affects the embedded version & MyODBC)
2001-12-06 13:10:51 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
Update copyright Fixed memory leak on shutdown (Affects the embedded version & MyODBC)
2001-12-06 13:10:51 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
/*****************************************************************************
** This file implements classes defined in field.h
*****************************************************************************/
Add ifdefs to control when "#pragma implementation" should be used Added some more ifdefs for "#pragma interface"
2005-05-26 12:09:14 +02:00
#ifdef USE_PRAGMA_IMPLEMENTATION
Import changeset
2000-07-31 21:29:14 +02:00
#pragma implementation // gcc: Class implementation
#endif
#include "mysql_priv.h"
#include "sql_select.h"
#include <m_ctype.h>
#include <errno.h>
#ifdef HAVE_FCONVERT
#include <floatingpoint.h>
#endif
Decimal field fix overflow
2002-10-30 17:39:52 +01:00
// Maximum allowed exponent value for converting string to decimal
#define MAX_EXPONENT 1024
Import changeset
2000-07-31 21:29:14 +02:00
/*****************************************************************************
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Instansiate templates and static variables
Import changeset
2000-07-31 21:29:14 +02:00
*****************************************************************************/
WL#2286 - Compile MySQL w/YASSL support Fix GCC 4.0 link failure. Better CXX_VERSION guessing.
2005-06-22 11:08:28 +02:00
#ifdef HAVE_EXPLICIT_TEMPLATE_INSTANTIATION
Import changeset
2000-07-31 21:29:14 +02:00
template class List<create_field>;
template class List_iterator<create_field>;
#endif
uchar Field_null::null[1]={1};
const char field_separator=',';
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
#define DOUBLE_TO_STRING_CONVERSION_BUFFER_SIZE 320
A fix and a test case for Bug#9735. No separate typecode for MEDIUMTEXT/LONGTEXT is added, as we have no sound decision yet what typecodes and for what types are sent by the server (aka what constitutes a distinct type in MySQL).
2005-07-14 13:13:23 +02:00
#define BLOB_PACK_LENGTH_TO_MAX_LENGH(arg) \
((ulong) ((LL(1) << min(arg, 4) * 8) - LL(1)))
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
#define ASSERT_COLUMN_MARKED_FOR_READ DBUG_ASSERT(!table->read_set || bitmap_is_set(table->read_set, field_index))
#define ASSERT_COLUMN_MARKED_FOR_WRITE DBUG_ASSERT(!table->write_set || bitmap_is_set(table->write_set, field_index))
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
/*
Rules for merging different types of fields in UNION
NOTE: to avoid 256*256 table, gap in table types numeration is skiped
following #defines describe that gap and how to canculate number of fields
and index of field in thia array.
*/
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
#define FIELDTYPE_TEAR_FROM (MYSQL_TYPE_BIT + 1)
#define FIELDTYPE_TEAR_TO (MYSQL_TYPE_NEWDECIMAL - 1)
#define FIELDTYPE_NUM (FIELDTYPE_TEAR_FROM + (255 - FIELDTYPE_TEAR_TO))
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
inline int field_type2index (enum_field_types field_type)
{
return (field_type < FIELDTYPE_TEAR_FROM ?
field_type :
((int)FIELDTYPE_TEAR_FROM) + (field_type - FIELDTYPE_TEAR_TO) - 1);
}
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
static enum_field_types field_types_merge_rules [FIELDTYPE_NUM][FIELDTYPE_NUM]=
{
/* MYSQL_TYPE_DECIMAL -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_NEWDECIMAL,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_NEWDECIMAL,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_DECIMAL, MYSQL_TYPE_DECIMAL,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_TINY -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_TINY,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_SHORT, MYSQL_TYPE_LONG,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_FLOAT, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_TINY, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_INT24,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_SHORT -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_SHORT,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_SHORT, MYSQL_TYPE_LONG,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_FLOAT, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_SHORT, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_INT24,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_SHORT,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_LONG -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_LONG,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_LONG, MYSQL_TYPE_LONG,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_LONG, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_INT24,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_LONG,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_FLOAT -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_FLOAT,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_FLOAT, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_FLOAT, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_FLOAT, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_FLOAT, MYSQL_TYPE_INT24,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_FLOAT,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_DOUBLE -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_INT24,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_NULL -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_TINY,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_SHORT, MYSQL_TYPE_LONG,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_FLOAT, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_NULL, MYSQL_TYPE_TIMESTAMP,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_INT24,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_TIME,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_DATETIME, MYSQL_TYPE_YEAR,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_BIT,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_ENUM,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_SET, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_STRING, MYSQL_TYPE_GEOMETRY
},
/* MYSQL_TYPE_TIMESTAMP -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_TIMESTAMP, MYSQL_TYPE_TIMESTAMP,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_DATETIME, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_DATETIME, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_LONGLONG -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_LONGLONG,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_LONGLONG,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_LONG,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_LONGLONG,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_INT24 -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_INT24,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_INT24, MYSQL_TYPE_LONG,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_FLOAT, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_INT24, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_INT24,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_INT24,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_DATE -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_DATETIME, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_TIME -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_TIME, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_DATETIME, MYSQL_TYPE_TIME,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_DATETIME, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_DATETIME -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_DATETIME, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_DATETIME, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_DATETIME, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_YEAR -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_DECIMAL, MYSQL_TYPE_TINY,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_SHORT, MYSQL_TYPE_LONG,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_FLOAT, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_YEAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_LONGLONG, MYSQL_TYPE_INT24,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_YEAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_NEWDATE -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_DATETIME,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_DATETIME, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_NEWDATE, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
},
/* MYSQL_TYPE_VARCHAR -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR
},
/* MYSQL_TYPE_BIT -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_BIT, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_BIT,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
},
/* MYSQL_TYPE_NEWDECIMAL -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_NEWDECIMAL,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_NEWDECIMAL,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_DOUBLE, MYSQL_TYPE_DOUBLE,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_NEWDECIMAL,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_NEWDECIMAL,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_NEWDECIMAL, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_ENUM -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_ENUM, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_SET -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_SET, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_TINY_BLOB -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_BIT <16>-<245>
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
MYSQL_TYPE_TINY_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
MYSQL_TYPE_BLOB, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_TINY_BLOB, MYSQL_TYPE_TINY_BLOB
},
/* MYSQL_TYPE_MEDIUM_BLOB -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_BIT <16>-<245>
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
MYSQL_TYPE_MEDIUM_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_MEDIUM_BLOB
},
/* MYSQL_TYPE_LONG_BLOB -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BIT <16>-<245>
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
MYSQL_TYPE_LONG_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_LONG_BLOB, MYSQL_TYPE_LONG_BLOB
},
/* MYSQL_TYPE_BLOB -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_BIT <16>-<245>
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
MYSQL_TYPE_BLOB,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_BLOB, MYSQL_TYPE_BLOB
},
/* MYSQL_TYPE_VAR_STRING -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
},
/* MYSQL_TYPE_STRING -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_STRING, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
MYSQL_TYPE_STRING,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
MYSQL_TYPE_STRING, MYSQL_TYPE_TINY_BLOB,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_STRING, MYSQL_TYPE_STRING
},
/* MYSQL_TYPE_GEOMETRY -> */
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_GEOMETRY, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_BIT <16>-<245>
MYSQL_TYPE_VARCHAR,
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_VARCHAR, MYSQL_TYPE_TINY_BLOB,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
MYSQL_TYPE_MEDIUM_BLOB, MYSQL_TYPE_LONG_BLOB,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
MYSQL_TYPE_BLOB, MYSQL_TYPE_VARCHAR,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
MYSQL_TYPE_STRING, MYSQL_TYPE_GEOMETRY
}
};
/*
Return type of which can carry value of both given types in UNION result
SYNOPSIS
Field::field_type_merge()
a, b types for merging
RETURN
type of field
*/
enum_field_types Field::field_type_merge(enum_field_types a,
enum_field_types b)
{
DBUG_ASSERT(a < FIELDTYPE_TEAR_FROM || a > FIELDTYPE_TEAR_TO);
DBUG_ASSERT(b < FIELDTYPE_TEAR_FROM || b > FIELDTYPE_TEAR_TO);
return field_types_merge_rules[field_type2index(a)]
[field_type2index(b)];
}
static Item_result field_types_result_type [FIELDTYPE_NUM]=
{
//MYSQL_TYPE_DECIMAL MYSQL_TYPE_TINY
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
DECIMAL_RESULT, INT_RESULT,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SHORT MYSQL_TYPE_LONG
INT_RESULT, INT_RESULT,
//MYSQL_TYPE_FLOAT MYSQL_TYPE_DOUBLE
REAL_RESULT, REAL_RESULT,
//MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
STRING_RESULT, STRING_RESULT,
//MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
INT_RESULT, INT_RESULT,
//MYSQL_TYPE_DATE MYSQL_TYPE_TIME
STRING_RESULT, STRING_RESULT,
//MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
STRING_RESULT, INT_RESULT,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDATE MYSQL_TYPE_VARCHAR
STRING_RESULT, STRING_RESULT,
//MYSQL_TYPE_BIT <16>-<245>
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
STRING_RESULT,
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
//MYSQL_TYPE_NEWDECIMAL MYSQL_TYPE_ENUM
DECIMAL_RESULT, STRING_RESULT,
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
//MYSQL_TYPE_SET MYSQL_TYPE_TINY_BLOB
STRING_RESULT, STRING_RESULT,
//MYSQL_TYPE_MEDIUM_BLOB MYSQL_TYPE_LONG_BLOB
STRING_RESULT, STRING_RESULT,
//MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
STRING_RESULT, STRING_RESULT,
//MYSQL_TYPE_STRING MYSQL_TYPE_GEOMETRY
STRING_RESULT, STRING_RESULT
};
/*
Detect Item_result by given field type of UNION merge result
SYNOPSIS
Field::result_merge_type()
field_type given field type
RETURN
Item_result (type of internal MySQL expression result)
*/
Item_result Field::result_merge_type(enum_field_types field_type)
{
DBUG_ASSERT(field_type < FIELDTYPE_TEAR_FROM || field_type
> FIELDTYPE_TEAR_TO);
return field_types_result_type[field_type2index(field_type)];
}
Import changeset
2000-07-31 21:29:14 +02:00
/*****************************************************************************
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Static help functions
Import changeset
2000-07-31 21:29:14 +02:00
*****************************************************************************/
Fix partial keys when converting VARCHAR to TEXT. (Bug #10543)
2005-05-26 03:11:47 +02:00
/*
Check whether a field type can be partially indexed by a key
This is a static method, rather than a virtual function, because we need
to check the type of a non-Field in mysql_alter_table().
SYNOPSIS
type_can_have_key_part()
type field type
RETURN
TRUE Type can have a prefixed key
FALSE Type can not have a prefixed key
*/
bool Field::type_can_have_key_part(enum enum_field_types type)
{
switch (type) {
case MYSQL_TYPE_VARCHAR:
case MYSQL_TYPE_TINY_BLOB:
case MYSQL_TYPE_MEDIUM_BLOB:
case MYSQL_TYPE_LONG_BLOB:
case MYSQL_TYPE_BLOB:
case MYSQL_TYPE_VAR_STRING:
case MYSQL_TYPE_STRING:
return TRUE;
default:
return FALSE;
}
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
/*
Numeric fields base class constructor
*/
Field_num::Field_num(char *ptr_arg,uint32 len_arg, uchar *null_ptr_arg,
uchar null_bit_arg, utype unireg_check_arg,
const char *field_name_arg,
uint8 dec_arg, bool zero_arg, bool unsigned_arg)
:Field(ptr_arg, len_arg, null_ptr_arg, null_bit_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check_arg, field_name_arg),
Precision Math implementation
2005-02-08 23:50:45 +01:00
dec(dec_arg),zerofill(zero_arg),unsigned_flag(unsigned_arg)
{
if (zerofill)
flags|=ZEROFILL_FLAG;
if (unsigned_flag)
flags|=UNSIGNED_FLAG;
}
Import changeset
2000-07-31 21:29:14 +02:00
void Field_num::prepend_zeros(String *value)
{
int diff;
if ((diff= (int) (field_length - value->length())) > 0)
{
bmove_upp((char*) value->ptr()+field_length,value->ptr()+value->length(),
value->length());
bfill((char*) value->ptr(),diff,'0');
value->length(field_length);
(void) value->c_ptr_quick(); // Avoid warnings in purify
}
}
/*
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Test if given number is a int (or a fixed format float with .000)
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
SYNOPSIS
test_if_int()
str String to test
end Pointer to char after last used digit
cs Character set
NOTES
This is called after one has called my_strntol() or similar function.
This is only used to give warnings in ALTER TABLE or LOAD DATA...
TODO
Make this multi-byte-character safe
RETURN
Precision Math implementation
2005-02-08 23:50:45 +01:00
0 OK
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
1 error. A warning is pushed if field_name != 0
Import changeset
2000-07-31 21:29:14 +02:00
*/
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
bool Field::check_int(const char *str, int length, const char *int_end,
CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
const char *end;
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
if (str == int_end)
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
{
char buff[128];
String tmp(buff,(uint32) sizeof(buff), system_charset_info);
tmp.copy(str, length, system_charset_info);
push_warning_printf(table->in_use, MYSQL_ERROR::WARN_LEVEL_WARN,
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD,
ER(ER_TRUNCATED_WRONG_VALUE_FOR_FIELD),
"integer", tmp.c_ptr(), field_name,
(ulong) table->in_use->row_count);
return 1; // Empty string
}
end= str+length;
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
if ((str= int_end) == end)
Precision Math implementation
2005-02-08 23:50:45 +01:00
return 0; // OK; All digits was used
Import changeset
2000-07-31 21:29:14 +02:00
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
/* Allow end .0000 */
if (*str == '.')
Import changeset
2000-07-31 21:29:14 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
for (str++ ; str != end && *str == '0'; str++)
;
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
}
/* Allow end space */
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
for ( ; str != end ; str++)
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
{
if (!my_isspace(cs,*str))
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
{
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
return 1;
}
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
/*
Process decimal library return codes and issue warnings for overflow and
truncation.
SYNOPSIS
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
Field::warn_if_overflow()
Precision Math implementation
2005-02-08 23:50:45 +01:00
op_result decimal library return code (E_DEC_* see include/decimal.h)
RETURN
1 there was overflow
0 no error or some other errors except overflow
*/
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
int Field::warn_if_overflow(int op_result)
Precision Math implementation
2005-02-08 23:50:45 +01:00
{
if (op_result == E_DEC_OVERFLOW)
{
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
return 1;
}
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
if (op_result == E_DEC_TRUNCATED)
{
Precision Math implementation
2005-02-08 23:50:45 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_NOTE, WARN_DATA_TRUNCATED, 1);
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
/* We return 0 here as this is not a critical issue */
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
return 0;
}
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
warnings removed
2004-03-16 16:35:49 +01:00
#ifdef NOT_USED
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
static bool test_if_real(const char *str,int length, CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
cs= system_charset_info; // QQ move test_if_real into CHARSET_INFO struct
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
while (length && my_isspace(cs,*str))
Import changeset
2000-07-31 21:29:14 +02:00
{ // Allow start space
length--; str++;
}
if (!length)
return 0;
if (*str == '+' || *str == '-')
{
length--; str++;
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
if (!length || !(my_isdigit(cs,*str) || *str == '.'))
Import changeset
2000-07-31 21:29:14 +02:00
return 0;
}
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
while (length && my_isdigit(cs,*str))
Import changeset
2000-07-31 21:29:14 +02:00
{
length--; str++;
}
if (!length)
return 1;
if (*str == '.')
{
length--; str++;
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
while (length && my_isdigit(cs,*str))
Import changeset
2000-07-31 21:29:14 +02:00
{
length--; str++;
}
}
if (!length)
return 1;
if (*str == 'E' || *str == 'e')
{
New ctype functions/macros to support many charsets at a time
2002-03-12 18:37:58 +01:00
if (length < 3 || (str[1] != '+' && str[1] != '-') ||
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
!my_isdigit(cs,str[2]))
Import changeset
2000-07-31 21:29:14 +02:00
return 0;
length-=3;
str+=3;
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
while (length && my_isdigit(cs,*str))
Import changeset
2000-07-31 21:29:14 +02:00
{
length--; str++;
}
}
Big code cleanup/review before 4.0.2 release. (All commit emails since 4.0.1 checked) This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
2002-06-11 10:20:31 +02:00
for (; length ; length--, str++)
Import changeset
2000-07-31 21:29:14 +02:00
{ // Allow end space
Preparation to charset dependant test_if_int and test_if_real
2002-12-11 14:42:48 +01:00
if (!my_isspace(cs,*str))
Import changeset
2000-07-31 21:29:14 +02:00
return 0;
}
return 1;
}
warnings removed
2004-03-16 16:35:49 +01:00
#endif
Import changeset
2000-07-31 21:29:14 +02:00
Fixed problem when comparing a key for a multi-byte-character set. (bug 152) Use 0x.... as strings if 'new' mode. (bug 152) Don't report -max on windows when InnoDB is enabled. (bug 332) Reset current_linfo; This could cause a hang when doing PURGE LOGS. Fix for row numbers in EXPLAIN (bug 322) Fix that USE_FRM works for all table types (bug 97)
2003-04-27 21:12:08 +02:00
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
/*
Interpret field value as an integer but return the result as a string.
This is used for printing bit_fields as numbers while debugging
*/
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
String *Field::val_int_as_str(String *val_buffer, my_bool unsigned_val)
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
CHARSET_INFO *cs= &my_charset_bin;
uint length= 21;
longlong value= val_int();
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
if (val_buffer->alloc(length))
return 0;
After merge fixes
2004-12-30 23:44:00 +01:00
length= (uint) (*cs->cset->longlong10_to_str)(cs, (char*) val_buffer->ptr(),
length,
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
unsigned_val ? 10 : -10,
After merge fixes
2004-12-30 23:44:00 +01:00
value);
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
val_buffer->length(length);
return val_buffer;
}
Import changeset
2000-07-31 21:29:14 +02:00
Field::Field(char *ptr_arg,uint32 length_arg,uchar *null_ptr_arg,
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uchar null_bit_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
utype unireg_check_arg, const char *field_name_arg)
Remove dflt_field from field structure as this was only needed when createing temporary table and I found another soultion that doesn't increase the size of the field structure for all table instances. (Better fix for bug #19089) Fixed compiler warnings Fixed valgrind warning in Item_date_add_intervall::eq. (Recoding of bugfix #19490)
2006-05-24 10:56:59 +02:00
:ptr(ptr_arg), null_ptr(null_ptr_arg),
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
table(0), orig_table(0), table_name(0),
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
field_name(field_name_arg),
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
key_start(0), part_of_key(0), part_of_key_not_clustered(0),
part_of_sortkey(0), unireg_check(unireg_check_arg),
Merge from 5.0
2006-05-31 10:18:58 +02:00
field_length(length_arg), null_bit(null_bit_arg)
Import changeset
2000-07-31 21:29:14 +02:00
{
flags=null_ptr ? 0: NOT_NULL_FLAG;
Changes for new binary .frm format Fixes after last merge from 4.0. (Code not yet complete, need anoter merge from 4.0)
2002-06-04 07:23:57 +02:00
comment.str= (char*) "";
comment.length=0;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
field_index= 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
uint Field::offset()
{
return (uint) (ptr - (char*) table->record[0]);
}
void Field::copy_from_tmp(int row_offset)
{
memcpy(ptr,ptr+row_offset,pack_length());
if (null_ptr)
{
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*null_ptr= (uchar) ((null_ptr[0] & (uchar) ~(uint) null_bit) |
null_ptr[row_offset] & (uchar) null_bit);
Import changeset
2000-07-31 21:29:14 +02:00
}
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field::send_binary(Protocol *protocol)
Import changeset
2000-07-31 21:29:14 +02:00
{
char buff[MAX_FIELD_WIDTH];
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
String tmp(buff,sizeof(buff),charset());
::reset(), HA_FAST_KEY_READ, disable_indexes(), enable_indexes(), start_bulk_insert(), end_bulk_insert() Field::val_str simplification, comment
2004-04-06 21:35:26 +02:00
val_str(&tmp);
Preparing to client<->server autoconvert
2003-03-17 10:14:04 +01:00
return protocol->store(tmp.ptr(), tmp.length(), tmp.charset());
Import changeset
2000-07-31 21:29:14 +02:00
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal *Field::val_decimal(my_decimal *decimal)
{
/* This never have to be called */
DBUG_ASSERT(0);
return 0;
}
Import changeset
2000-07-31 21:29:14 +02:00
void Field_num::add_zerofill_and_unsigned(String &res) const
{
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
if (unsigned_flag)
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN(" unsigned"));
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
if (zerofill)
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN(" zerofill"));
Import changeset
2000-07-31 21:29:14 +02:00
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
void Field::make_field(Send_field *field)
Import changeset
2000-07-31 21:29:14 +02:00
{
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
if (orig_table->s->db.str && *orig_table->s->db.str)
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
{
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
field->db_name= orig_table->s->db.str;
field->org_table_name= orig_table->s->table_name.str;
fixed union types merging and table related metadata (BUG#8824)
2005-03-23 07:36:48 +01:00
}
else
postmerge 4.1->5.0 fixes
2005-04-01 01:14:30 +02:00
field->org_table_name= field->db_name= "";
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
field->table_name= orig_table->alias;
field->col_name= field->org_col_name= field_name;
Added charset number to result header
2003-02-04 02:19:19 +01:00
field->charsetnr= charset()->number;
Import changeset
2000-07-31 21:29:14 +02:00
field->length=field_length;
field->type=type();
field->flags=table->maybe_null ? (flags & ~NOT_NULL_FLAG) : flags;
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
field->decimals= 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
/*
Conversion from decimal to longlong with checking overflow and
setting correct value (min/max) in case of overflow
SYNOPSIS
Field::convert_decimal2longlong()
val value which have to be converted
unsigned_flag type of integer in which we convert val
err variable to pass error code
RETURN
value converted from val
*/
longlong Field::convert_decimal2longlong(const my_decimal *val,
bool unsigned_flag, int *err)
{
longlong i;
if (unsigned_flag)
{
if (val->sign())
{
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
i= 0;
*err= 1;
}
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
else if (warn_if_overflow(my_decimal2int(E_DEC_ERROR &
Precision Math implementation
2005-02-08 23:50:45 +01:00
~E_DEC_OVERFLOW & ~E_DEC_TRUNCATED,
val, TRUE, &i)))
{
i= ~(longlong) 0;
*err= 1;
}
}
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
else if (warn_if_overflow(my_decimal2int(E_DEC_ERROR &
Precision Math implementation
2005-02-08 23:50:45 +01:00
~E_DEC_OVERFLOW & ~E_DEC_TRUNCATED,
val, FALSE, &i)))
{
i= (val->sign() ? LONGLONG_MIN : LONGLONG_MAX);
*err= 1;
}
return i;
}
/*
Storing decimal in integer fields.
SYNOPSIS
Field_num::store_decimal()
val value for storing
NOTE
This method is used by all integer fields, real/decimal redefine it
RETURN
0 OK
!= 0 error
*/
int Field_num::store_decimal(const my_decimal *val)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Precision Math implementation
2005-02-08 23:50:45 +01:00
int err= 0;
longlong i= convert_decimal2longlong(val, unsigned_flag, &err);
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
return test(err | store(i, unsigned_flag));
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
/*
Return decimal value of integer field
SYNOPSIS
Field_num::val_decimal()
decimal_value buffer for storing decimal value
NOTE
This method is used by all integer fields, real/decimal redefine it
All longlong values fit in our decimal buffer which cal store 8*9=72
digits of integer number
RETURN
pointer to decimal buffer with value of field
*/
my_decimal* Field_num::val_decimal(my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
DBUG_ASSERT(result_type() == INT_RESULT);
longlong nr= val_int();
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
int2my_decimal(E_DEC_FATAL_ERROR, nr, unsigned_flag, decimal_value);
Precision Math implementation
2005-02-08 23:50:45 +01:00
return decimal_value;
}
Field_str::Field_str(char *ptr_arg,uint32 len_arg, uchar *null_ptr_arg,
uchar null_bit_arg, utype unireg_check_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
const char *field_name_arg, CHARSET_INFO *charset)
Precision Math implementation
2005-02-08 23:50:45 +01:00
:Field(ptr_arg, len_arg, null_ptr_arg, null_bit_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check_arg, field_name_arg)
Precision Math implementation
2005-02-08 23:50:45 +01:00
{
field_charset=charset;
if (charset->state & MY_CS_BINSORT)
flags|=BINARY_FLAG;
}
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
void Field_num::make_field(Send_field *field)
Import changeset
2000-07-31 21:29:14 +02:00
{
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
Field::make_field(field);
field->decimals= dec;
Import changeset
2000-07-31 21:29:14 +02:00
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
/*
Decimal representation of Field_str
SYNOPSIS
Field_str::store_decimal()
d value for storing
NOTE
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
Field_str is the base class for fields like Field_enum, Field_date and some
Precision Math implementation
2005-02-08 23:50:45 +01:00
similar. Some dates use fraction and also string value should be
converted to floating point value according our rules, so we use double
to store value of decimal in string
RETURN
0 OK
!= 0 error
*/
int Field_str::store_decimal(const my_decimal *d)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Precision Math implementation
2005-02-08 23:50:45 +01:00
double val;
/* TODO: use decimal2string? */
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
int err= warn_if_overflow(my_decimal2double(E_DEC_FATAL_ERROR &
Precision Math implementation
2005-02-08 23:50:45 +01:00
~E_DEC_OVERFLOW, d, &val));
return err | store(val);
}
my_decimal *Field_str::val_decimal(my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
longlong nr= val_int();
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
int2my_decimal(E_DEC_FATAL_ERROR, nr, 0, decimal_value);
Precision Math implementation
2005-02-08 23:50:45 +01:00
return decimal_value;
}
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
uint Field::fill_cache_field(CACHE_FIELD *copy)
{
Cleanup during review of new code Fixed wrong allocation that could cause buffer overrun when using join cache
2005-06-21 17:18:58 +02:00
uint store_length;
Import changeset
2000-07-31 21:29:14 +02:00
copy->str=ptr;
copy->length=pack_length();
copy->blob_field=0;
if (flags & BLOB_FLAG)
{
copy->blob_field=(Field_blob*) this;
copy->strip=0;
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
copy->length-= table->s->blob_ptr_size;
Import changeset
2000-07-31 21:29:14 +02:00
return copy->length;
}
Update results for new varchar handling Fixed compiler warnings String results in CREATE ... SELECT are now created as CHAR(0), VARCHAR(X) or TEXT() depending on item->max_length
2004-12-07 14:47:00 +01:00
else if (!zero_pack() &&
(type() == MYSQL_TYPE_STRING && copy->length >= 4 &&
copy->length < 256))
Cleanup during review of new code Fixed wrong allocation that could cause buffer overrun when using join cache
2005-06-21 17:18:58 +02:00
{
Import changeset
2000-07-31 21:29:14 +02:00
copy->strip=1; /* Remove end space */
Cleanup during review of new code Fixed wrong allocation that could cause buffer overrun when using join cache
2005-06-21 17:18:58 +02:00
store_length= 2;
}
Import changeset
2000-07-31 21:29:14 +02:00
else
Cleanup during review of new code Fixed wrong allocation that could cause buffer overrun when using join cache
2005-06-21 17:18:58 +02:00
{
Import changeset
2000-07-31 21:29:14 +02:00
copy->strip=0;
Cleanup during review of new code Fixed wrong allocation that could cause buffer overrun when using join cache
2005-06-21 17:18:58 +02:00
store_length= 0;
}
return copy->length+ store_length;
Import changeset
2000-07-31 21:29:14 +02:00
}
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
bool Field::get_date(TIME *ltime,uint fuzzydate)
Import changeset
2000-07-31 21:29:14 +02:00
{
char buff[40];
::reset(), HA_FAST_KEY_READ, disable_indexes(), enable_indexes(), start_bulk_insert(), end_bulk_insert() Field::val_str simplification, comment
2004-04-06 21:35:26 +02:00
String tmp(buff,sizeof(buff),&my_charset_bin),*res;
if (!(res=val_str(&tmp)) ||
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
str_to_datetime_with_warn(res->ptr(), res->length(),
ltime, fuzzydate) <= MYSQL_TIMESTAMP_ERROR)
Import changeset
2000-07-31 21:29:14 +02:00
return 1;
return 0;
}
bool Field::get_time(TIME *ltime)
{
char buff[40];
::reset(), HA_FAST_KEY_READ, disable_indexes(), enable_indexes(), start_bulk_insert(), end_bulk_insert() Field::val_str simplification, comment
2004-04-06 21:35:26 +02:00
String tmp(buff,sizeof(buff),&my_charset_bin),*res;
if (!(res=val_str(&tmp)) ||
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
str_to_time_with_warn(res->ptr(), res->length(), ltime))
Import changeset
2000-07-31 21:29:14 +02:00
return 1;
return 0;
}
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
/*
This is called when storing a date in a string
NOTES
Needs to be changed if/when we want to support different time formats
*/
Import changeset
2000-07-31 21:29:14 +02:00
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
int Field::store_time(TIME *ltime, timestamp_type type)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
A fix and test case for Bug#6049 "Loss of sign when using prepared statements and negative time/date values". The bug was in wrong sprintf format used in the client library. The fix moves TIME -> string conversion functions to sql-common and utilized them in the client library.
2004-10-15 22:12:59 +02:00
char buff[MAX_DATE_STRING_REP_LENGTH];
uint length= (uint) my_TIME_to_str(ltime, buff);
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
return store(buff, length, &my_charset_bin);
Import changeset
2000-07-31 21:29:14 +02:00
}
New handler::index_flags() definition to make it easy to check the full used key and a specific key part. Added key part to optimize_range() to fix problems when using fields in key parts.
2004-07-08 14:45:25 +02:00
bool Field::optimize_range(uint idx, uint part)
Import changeset
2000-07-31 21:29:14 +02:00
{
New handler::index_flags() definition to make it easy to check the full used key and a specific key part. Added key part to optimize_range() to fix problems when using fields in key parts.
2004-07-08 14:45:25 +02:00
return test(table->file->index_flags(idx, part, 1) & HA_READ_RANGE);
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Field *Field::new_field(MEM_ROOT *root, struct st_table *new_table)
{
Field *tmp;
if (!(tmp= (Field*) memdup_root(root,(char*) this,size_of())))
return 0;
if (tmp->table->maybe_null)
tmp->flags&= ~NOT_NULL_FLAG;
tmp->table= new_table;
tmp->key_start.init(0);
tmp->part_of_key.init(0);
tmp->part_of_sortkey.init(0);
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
tmp->unireg_check= Field::NONE;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
tmp->flags&= (NOT_NULL_FLAG | BLOB_FLAG | UNSIGNED_FLAG |
ZEROFILL_FLAG | BINARY_FLAG | ENUM_FLAG | SET_FLAG);
tmp->reset_fields();
return tmp;
}
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
Field *Field::new_key_field(MEM_ROOT *root, struct st_table *new_table,
char *new_ptr, uchar *new_null_ptr,
uint new_null_bit)
{
Field *tmp;
if ((tmp= new_field(root, new_table)))
{
tmp->ptr= new_ptr;
tmp->null_ptr= new_null_ptr;
tmp->null_bit= new_null_bit;
}
return tmp;
}
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
/*
SYNOPSIS
Field::quote_data()
unquoted_string Pointer pointing to the value of a field
DESCRIPTION
Simple method that passes the field type to the method "type_quote"
To get a true/false value as to whether the value in string1 needs
to be enclosed with quotes. This ensures that values in the final
sql statement to be passed to the remote server will be quoted properly
RETURN_VALUE
void Immediately - if string doesn't need quote
void Upon prepending/appending quotes on each side of variable
*/
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
bool Field::quote_data(String *unquoted_string)
{
char escaped_string[IO_SIZE];
DBUG_ENTER("Field::quote_data");
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
Field::quote_data(): don't call escape_string_for_mysql() unnecesary don't overwrite local buffer escape_string_for_mysql(): take a length of the destination buffer as an argument
2005-03-17 12:27:45 +01:00
if (!needs_quotes())
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
DBUG_RETURN(0);
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
Field::quote_data(): don't call escape_string_for_mysql() unnecesary don't overwrite local buffer escape_string_for_mysql(): take a length of the destination buffer as an argument
2005-03-17 12:27:45 +01:00
// this is the same call that mysql_real_escape_string() calls
if (escape_string_for_mysql(&my_charset_bin, (char *)escaped_string,
sizeof(escaped_string), unquoted_string->ptr(),
unquoted_string->length()) == (ulong)~0)
DBUG_RETURN(1);
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
// reset string, then re-append with quotes and escaped values
unquoted_string->length(0);
Field::quote_data(): don't call escape_string_for_mysql() unnecesary don't overwrite local buffer escape_string_for_mysql(): take a length of the destination buffer as an argument
2005-03-17 12:27:45 +01:00
if (unquoted_string->append('\'') ||
unquoted_string->append((char *)escaped_string) ||
unquoted_string->append('\''))
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
DBUG_RETURN(1);
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
DBUG_RETURN(0);
}
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
/*
Quote a field type if needed
SYNOPSIS
Field::type_quote
DESCRIPTION
Simple method to give true/false whether a field should be quoted.
Used when constructing INSERT and UPDATE queries to the remote server
see write_row and update_row
RETURN VALUE
0 if value is of type NOT needing quotes
1 if value is of type needing quotes
*/
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
bool Field::needs_quotes(void)
{
DBUG_ENTER("Field::type_quote");
Remove compiler warnings and remove not used variables (Found during build process)
2005-02-25 15:53:22 +01:00
switch (type()) {
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
//FIX this when kernel is fixed
case MYSQL_TYPE_VARCHAR :
case FIELD_TYPE_STRING :
case FIELD_TYPE_VAR_STRING :
case FIELD_TYPE_YEAR :
case FIELD_TYPE_NEWDATE :
case FIELD_TYPE_TIME :
case FIELD_TYPE_TIMESTAMP :
case FIELD_TYPE_DATE :
case FIELD_TYPE_DATETIME :
case FIELD_TYPE_TINY_BLOB :
case FIELD_TYPE_BLOB :
case FIELD_TYPE_MEDIUM_BLOB :
case FIELD_TYPE_LONG_BLOB :
case FIELD_TYPE_GEOMETRY :
BUG #14532 Added FIELD_TYPE_BIT to field method 'needs_quotes' to make BIT columns work.
2005-11-01 02:17:16 +01:00
case FIELD_TYPE_BIT:
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
DBUG_RETURN(1);
case FIELD_TYPE_DECIMAL :
case FIELD_TYPE_TINY :
case FIELD_TYPE_SHORT :
case FIELD_TYPE_INT24 :
case FIELD_TYPE_LONG :
case FIELD_TYPE_FLOAT :
case FIELD_TYPE_DOUBLE :
case FIELD_TYPE_LONGLONG :
case FIELD_TYPE_NULL :
case FIELD_TYPE_SET :
case FIELD_TYPE_ENUM :
DBUG_RETURN(0);
Remove compiler warnings and remove not used variables (Found during build process)
2005-02-25 15:53:22 +01:00
default:
DBUG_RETURN(0);
-Added quote_data and needs_quotes (moved from federated handler. -New tests and results logging_ok: Logging to logging@openlogging.org accepted ha_federated.h: removed quote_data and type_quote (now in the Field class) ha_federated.cc: moved quote_data and type_quote to field class field.h: new methods quote_data and needs_quotes declared field.cc: new field class methods quote_data and needs_quotes (per Monty's request) federated.test: more tests, joins, index tests have_federated_db.require: new name of federated system var federated.result: new test results for federated handler have_federated_db.inc: changed name of variable in test due to change in vars sql_analyse.cc: over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler. mysql_priv.h: define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler ha_federated.cc: implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.
2005-01-21 03:36:40 +01:00
}
}
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
/* This is used to generate a field in TABLE from TABLE_SHARE */
Field *Field::clone(MEM_ROOT *root, struct st_table *new_table)
{
Field *tmp;
if ((tmp= (Field*) memdup_root(root,(char*) this,size_of())))
{
tmp->init(new_table);
tmp->move_field_offset((my_ptrdiff_t) (new_table->record[0] -
new_table->s->default_values));
}
return tmp;
}
Move all sql_type() to the same file
2002-12-17 08:25:27 +01:00
/****************************************************************************
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Field_null, a field that always return NULL
Move all sql_type() to the same file
2002-12-17 08:25:27 +01:00
****************************************************************************/
void Field_null::sql_type(String &res) const
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("null"));
Move all sql_type() to the same file
2002-12-17 08:25:27 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
Indentation cleanup & new comments
2002-06-28 08:18:23 +02:00
Functions for the Field_decimal class
This is an number stored as a pre-space (or pre-zero) string
Import changeset
2000-07-31 21:29:14 +02:00
****************************************************************************/
void
Field_decimal::reset(void)
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
Field_decimal::store(STRING_WITH_LEN("0"),&my_charset_bin);
Import changeset
2000-07-31 21:29:14 +02:00
}
void Field_decimal::overflow(bool negative)
{
uint len=field_length;
Fixed problems with DECIMAL() type on overflow.
2002-05-02 17:04:21 +02:00
char *to=ptr, filler= '9';
Indentation cleanup & new comments
2002-06-28 08:18:23 +02:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Fixed problems with DECIMAL() type on overflow.
2002-05-02 17:04:21 +02:00
if (negative)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixed problems with DECIMAL() type on overflow.
2002-05-02 17:04:21 +02:00
if (!unsigned_flag)
{
/* Put - sign as a first digit so we'll have -999..999 or 999..999 */
*to++ = '-';
len--;
}
else
{
filler= '0'; // Fill up with 0
if (!zerofill)
{
/*
Handle unsigned integer without zerofill, in which case
the number should be of format ' 0' or ' 0.000'
*/
uint whole_part=field_length- (dec ? dec+2 : 1);
// Fill with spaces up to the first digit
bfill(to, whole_part, ' ');
to+= whole_part;
len-= whole_part;
// The main code will also handle the 0 before the decimal point
}
}
Import changeset
2000-07-31 21:29:14 +02:00
}
Fixed problems with DECIMAL() type on overflow.
2002-05-02 17:04:21 +02:00
bfill(to, len, filler);
Import changeset
2000-07-31 21:29:14 +02:00
if (dec)
ptr[field_length-dec-1]='.';
return;
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_decimal::store(const char *from, uint len, CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
String tmp(buff,sizeof(buff), &my_charset_bin);
removed several cases of default_charset_info use a bit more ucs2 compatibility
2002-12-19 06:38:29 +01:00
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
/* Convert character set if the old one is multi byte */
if (cs->mbmaxlen > 1)
{
A fix according to Monty's request: "uint *errors" is now a non-optional parameter in String:copy() and copy_and_convert().
2004-10-29 14:00:39 +02:00
uint dummy_errors;
tmp.copy(from, len, cs, &my_charset_bin, &dummy_errors);
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
from= tmp.ptr();
len= tmp.length();
}
removed several cases of default_charset_info use a bit more ucs2 compatibility
2002-12-19 06:38:29 +01:00
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
const char *end= from+len;
/* The pointer where the field value starts (i.e., "where to write") */
char *to=ptr;
uint tmp_dec, tmp_uint;
/*
The sign of the number : will be 0 (means positive but sign not
specified), '+' or '-'
*/
char sign_char=0;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/* The pointers where prezeros start and stop */
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
const char *pre_zeros_from, *pre_zeros_end;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/* The pointers where digits at the left of '.' start and stop */
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
const char *int_digits_from, *int_digits_end;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/* The pointers where digits at the right of '.' start and stop */
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
const char *frac_digits_from, *frac_digits_end;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/* The sign of the exponent : will be 0 (means no exponent), '+' or '-' */
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
char expo_sign_char=0;
Decimal field fix overflow
2002-10-30 17:39:52 +01:00
uint exponent=0; // value of the exponent
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
/*
Pointers used when digits move from the left of the '.' to the
right of the '.' (explained below)
*/
const char *int_digits_tail_from;
/* Number of 0 that need to be added at the left of the '.' (1E3: 3 zeros) */
uint int_digits_added_zeros;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/*
Pointer used when digits move from the right of the '.' to the left
of the '.'
*/
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
const char *frac_digits_head_end;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/* Number of 0 that need to be added at the right of the '.' (for 1E-3) */
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
uint frac_digits_added_zeros;
char *pos,*tmp_left_pos,*tmp_right_pos;
/* Pointers that are used as limits (begin and end of the field buffer) */
char *left_wall,*right_wall;
char tmp_char;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/*
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
To remember if table->in_use->cuted_fields has already been incremented,
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
to do that only once
*/
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
bool is_cuted_fields_incr=0;
LINT_INIT(int_digits_tail_from);
LINT_INIT(int_digits_added_zeros);
LINT_INIT(frac_digits_head_end);
LINT_INIT(frac_digits_added_zeros);
/*
There are three steps in this function :
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
- parse the input string
- modify the position of digits around the decimal dot '.'
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
according to the exponent value (if specified)
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
- write the formatted number
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
*/
if ((tmp_dec=dec))
tmp_dec++;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
/* skip pre-space */
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
while (from != end && my_isspace(&my_charset_bin,*from))
Import changeset
2000-07-31 21:29:14 +02:00
from++;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
if (from == end)
Import changeset
2000-07-31 21:29:14 +02:00
{
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
is_cuted_fields_incr=1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
else if (*from == '+' || *from == '-') // Found some sign ?
Import changeset
2000-07-31 21:29:14 +02:00
{
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
sign_char= *from++;
Indentation cleanup & new comments
2002-06-28 08:18:23 +02:00
/*
Allow storing "+456" then decimal declared unsigned
2002-07-30 17:52:51 +02:00
We allow "+" for unsigned decimal unless defined different
Both options allowed as one may wish not to have "+" for unsigned numbers
because of data processing issues
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
*/
if (unsigned_flag)
{
if (sign_char=='-')
Fixed problems with DECIMAL() type on overflow.
2002-05-02 17:04:21 +02:00
{
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Field_decimal::overflow(1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
return 1;
Fixed problems with DECIMAL() type on overflow.
2002-05-02 17:04:21 +02:00
}
Allow storing "+456" then decimal declared unsigned
2002-07-30 17:52:51 +02:00
/*
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
Defining this will not store "+" for unsigned decimal type even if
it is passed in numeric string. This will make some tests to fail
Allow storing "+456" then decimal declared unsigned
2002-07-30 17:52:51 +02:00
*/
#ifdef DONT_ALLOW_UNSIGNED_PLUS
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
else
sign_char=0;
Allow storing "+456" then decimal declared unsigned
2002-07-30 17:52:51 +02:00
#endif
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
Indentation cleanup & new comments
2002-06-28 08:18:23 +02:00
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
pre_zeros_from= from;
for (; from!=end && *from == '0'; from++) ; // Read prezeros
pre_zeros_end=int_digits_from=from;
/* Read non zero digits at the left of '.'*/
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
for (; from != end && my_isdigit(&my_charset_bin, *from) ; from++) ;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
int_digits_end=from;
if (from!=end && *from == '.') // Some '.' ?
from++;
frac_digits_from= from;
/* Read digits at the right of '.' */
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
for (;from!=end && my_isdigit(&my_charset_bin, *from); from++) ;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
frac_digits_end=from;
// Some exponentiation symbol ?
if (from != end && (*from == 'e' || *from == 'E'))
{
from++;
if (from != end && (*from == '+' || *from == '-')) // Some exponent sign ?
expo_sign_char= *from++;
else
expo_sign_char= '+';
/*
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
Read digits of the exponent and compute its value. We must care about
'exponent' overflow, because as unsigned arithmetic is "modulo", big
exponents will become small (e.g. 1e4294967296 will become 1e0, and the
field will finally contain 1 instead of its max possible value).
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
*/
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
for (;from!=end && my_isdigit(&my_charset_bin, *from); from++)
Fix exponent overflow handling for decimal type. Overflow lead to crash
2002-10-13 19:11:01 +02:00
{
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
exponent=10*exponent+(*from-'0');
Decimal field fix overflow
2002-10-30 17:39:52 +01:00
if (exponent>MAX_EXPONENT)
Fix exponent overflow handling for decimal type. Overflow lead to crash
2002-10-13 19:11:01 +02:00
break;
}
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
/*
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
We only have to generate warnings if count_cuted_fields is set.
This is to avoid extra checks of the number when they are not needed.
Precision Math implementation
2005-02-08 23:50:45 +01:00
Even if this flag is not set, it's OK to increment warnings, if
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
it makes the code easer to read.
Import changeset
2000-07-31 21:29:14 +02:00
*/
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if (table->in_use->count_cuted_fields)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
// Skip end spaces
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
for (;from != end && my_isspace(&my_charset_bin, *from); from++) ;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
if (from != end) // If still something left, warn
Import changeset
2000-07-31 21:29:14 +02:00
{
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
is_cuted_fields_incr=1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
/*
Now "move" digits around the decimal dot according to the exponent value,
and add necessary zeros.
Examples :
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
- 1E+3 : needs 3 more zeros at the left of '.' (int_digits_added_zeros=3)
- 1E-3 : '1' moves at the right of '.', and 2 more zeros are needed
between '.' and '1'
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
- 1234.5E-3 : '234' moves at the right of '.'
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
These moves are implemented with pointers which point at the begin
and end of each moved segment. Examples :
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
- 1234.5E-3 : before the code below is executed, the int_digits part is
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
from '1' to '4' and the frac_digits part from '5' to '5'. After the code
below, the int_digits part is from '1' to '1', the frac_digits_head
part is from '2' to '4', and the frac_digits part from '5' to '5'.
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
- 1234.5E3 : before the code below is executed, the int_digits part is
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
from '1' to '4' and the frac_digits part from '5' to '5'. After the code
below, the int_digits part is from '1' to '4', the int_digits_tail
part is from '5' to '5', the frac_digits part is empty, and
int_digits_added_zeros=2 (to make 1234500).
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
*/
Fix exponent overflow handling for decimal type. Overflow lead to crash
2002-10-13 19:11:01 +02:00
/*
Decimal field fix overflow
2002-10-30 17:39:52 +01:00
Below tmp_uint cannot overflow with small enough MAX_EXPONENT setting,
Fix exponent overflow handling for decimal type. Overflow lead to crash
2002-10-13 19:11:01 +02:00
as int_digits_added_zeros<=exponent<4G and
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
(int_digits_end-int_digits_from)<=max_allowed_packet<=2G and
(frac_digits_from-int_digits_tail_from)<=max_allowed_packet<=2G
Fix exponent overflow handling for decimal type. Overflow lead to crash
2002-10-13 19:11:01 +02:00
*/
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
if (!expo_sign_char)
tmp_uint=tmp_dec+(uint)(int_digits_end-int_digits_from);
else if (expo_sign_char == '-')
{
tmp_uint=min(exponent,(uint)(int_digits_end-int_digits_from));
frac_digits_added_zeros=exponent-tmp_uint;
int_digits_end -= tmp_uint;
frac_digits_head_end=int_digits_end+tmp_uint;
Decimal field fix overflow
2002-10-30 17:39:52 +01:00
tmp_uint=tmp_dec+(uint)(int_digits_end-int_digits_from);
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
else // (expo_sign_char=='+')
{
tmp_uint=min(exponent,(uint)(frac_digits_end-frac_digits_from));
int_digits_added_zeros=exponent-tmp_uint;
int_digits_tail_from=frac_digits_from;
frac_digits_from=frac_digits_from+tmp_uint;
/*
We "eat" the heading zeros of the
int_digits.int_digits_tail.int_digits_added_zeros concatenation
(for example 0.003e3 must become 3 and not 0003)
*/
if (int_digits_from == int_digits_end)
{
/*
There was nothing in the int_digits part, so continue
eating int_digits_tail zeros
*/
for (; int_digits_tail_from != frac_digits_from &&
*int_digits_tail_from == '0'; int_digits_tail_from++) ;
if (int_digits_tail_from == frac_digits_from)
{
// there were only zeros in int_digits_tail too
int_digits_added_zeros=0;
}
Import changeset
2000-07-31 21:29:14 +02:00
}
fixes for windows 64-bit compiler warnings
2005-06-13 12:41:15 +02:00
tmp_uint= (uint) (tmp_dec+(int_digits_end-int_digits_from)+
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
(uint)(frac_digits_from-int_digits_tail_from)+
int_digits_added_zeros);
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
/*
Now write the formated number
First the digits of the int_% parts.
Do we have enough room to write these digits ?
If the sign is defined and '-', we need one position for it
*/
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
if (field_length < tmp_uint + (int) (sign_char == '-'))
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
{
Import changeset
2000-07-31 21:29:14 +02:00
// too big number, change to max or min number
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Field_decimal::overflow(sign_char == '-');
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
/*
Tmp_left_pos is the position where the leftmost digit of
the int_% parts will be written
*/
tmp_left_pos=pos=to+(uint)(field_length-tmp_uint);
// Write all digits of the int_% parts
while (int_digits_from != int_digits_end)
*pos++ = *int_digits_from++ ;
if (expo_sign_char == '+')
{
while (int_digits_tail_from != frac_digits_from)
*pos++= *int_digits_tail_from++;
while (int_digits_added_zeros-- >0)
*pos++= '0';
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
/*
Note the position where the rightmost digit of the int_% parts has been
written (this is to later check if the int_% parts contained nothing,
meaning an extra 0 is needed).
*/
tmp_right_pos=pos;
Import changeset
2000-07-31 21:29:14 +02:00
/*
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Step back to the position of the leftmost digit of the int_% parts,
to write sign and fill with zeros or blanks or prezeros.
Import changeset
2000-07-31 21:29:14 +02:00
*/
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
pos=tmp_left_pos-1;
if (zerofill)
{
left_wall=to-1;
Fix for bug when zeros are to be prepended to decimal field
2003-11-18 14:58:27 +01:00
while (pos > left_wall) // Fill with zeros
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
*pos--='0';
}
else
{
Indentation cleanup & new comments
2002-06-28 08:18:23 +02:00
left_wall=to+(sign_char != 0)-1;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
if (!expo_sign_char) // If exponent was specified, ignore prezeros
{
Fix for bug when zeros are to be prepended to decimal field
2003-11-18 14:58:27 +01:00
for (;pos > left_wall && pre_zeros_from !=pre_zeros_end;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
pre_zeros_from++)
*pos--= '0';
}
if (pos == tmp_right_pos-1)
*pos--= '0'; // no 0 has ever been written, so write one
left_wall= to-1;
if (sign_char && pos != left_wall)
{
/* Write sign if possible (it is if sign is '-') */
*pos--= sign_char;
}
while (pos != left_wall)
*pos--=' '; //fill with blanks
}
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
/*
Write digits of the frac_% parts ;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Depending on table->in_use->count_cutted_fields, we may also want
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
to know if some non-zero tail of these parts will
be truncated (for example, 0.002->0.00 will generate a warning,
while 0.000->0.00 will not)
(and 0E1000000000 will not, while 1E-1000000000 will)
*/
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
pos=to+(uint)(field_length-tmp_dec); // Calculate post to '.'
right_wall=to+field_length;
if (pos != right_wall)
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
*pos++='.';
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
if (expo_sign_char == '-')
{
while (frac_digits_added_zeros-- > 0)
Import changeset
2000-07-31 21:29:14 +02:00
{
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
if (pos == right_wall)
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if (table->in_use->count_cuted_fields && !is_cuted_fields_incr)
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
break; // Go on below to see if we lose non zero digits
Merge with 4.0
2002-11-21 14:56:48 +01:00
return 0;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
*pos++='0';
}
while (int_digits_end != frac_digits_head_end)
{
tmp_char= *int_digits_end++;
if (pos == right_wall)
Import changeset
2000-07-31 21:29:14 +02:00
{
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
if (tmp_char != '0') // Losing a non zero digit ?
{
if (!is_cuted_fields_incr)
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
WARN_DATA_TRUNCATED, 1);
Merge with 4.0
2002-11-21 14:56:48 +01:00
return 0;
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
}
continue;
Import changeset
2000-07-31 21:29:14 +02:00
}
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
*pos++= tmp_char;
Import changeset
2000-07-31 21:29:14 +02:00
}
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
for (;frac_digits_from!=frac_digits_end;)
{
tmp_char= *frac_digits_from++;
if (pos == right_wall)
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
{
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
if (tmp_char != '0') // Losing a non zero digit ?
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
{
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
if (!is_cuted_fields_incr)
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
{
/*
This is a note, not a warning, as we don't want to abort
when we cut decimals in strict mode
*/
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_NOTE, WARN_DATA_TRUNCATED, 1);
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
}
Merge with 4.0
2002-11-21 14:56:48 +01:00
return 0;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
continue;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
*pos++= tmp_char;
Import changeset
2000-07-31 21:29:14 +02:00
}
Final decimal bug fix changeset (hope)
2002-11-06 10:35:16 +01:00
while (pos != right_wall)
*pos++='0'; // Fill with zeros at right of '.'
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_decimal::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag && nr < 0)
{
overflow(1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
Updated windows files (VC++ files and winmysqladmin). Portability fixes. Removed compiler warnings.
2002-06-28 16:26:11 +02:00
#ifdef HAVE_FINITE
if (!finite(nr)) // Handle infinity as special case
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
{
overflow(nr < 0.0);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
return 1;
Adding handling of numbers with exponent to decimal type.
2002-06-24 19:30:54 +02:00
}
Updated windows files (VC++ files and winmysqladmin). Portability fixes. Removed compiler warnings.
2002-06-28 16:26:11 +02:00
#endif
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
reg4 uint i,length;
char fyllchar,*to;
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
char buff[DOUBLE_TO_STRING_CONVERSION_BUFFER_SIZE];
Import changeset
2000-07-31 21:29:14 +02:00
fyllchar = zerofill ? (char) '0' : (char) ' ';
snprintf compatibility fix
2003-08-12 15:28:36 +02:00
#ifdef HAVE_SNPRINTF
Fix for snprintf
2000-08-21 23:55:45 +02:00
buff[sizeof(buff)-1]=0; // Safety
snprintf(buff,sizeof(buff)-1, "%.*f",(int) dec,nr);
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
length=(uint) strlen(buff);
FLUSH TABLE table_list Fixes for RENAME TABLE Portability fixes
2000-08-21 23:18:32 +02:00
#else
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
length=(uint) my_sprintf(buff,(buff,"%.*f",dec,nr));
FLUSH TABLE table_list Fixes for RENAME TABLE Portability fixes
2000-08-21 23:18:32 +02:00
#endif
Import changeset
2000-07-31 21:29:14 +02:00
if (length > field_length)
{
overflow(nr < 0.0);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
to=ptr;
for (i=field_length-length ; i-- > 0 ;)
*to++ = fyllchar;
memcpy(to,buff,length);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_decimal::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
char buff[22];
uint length, int_part;
char fyllchar, *to;
if (nr < 0 && unsigned_flag && !unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
overflow(1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
length= (uint) (longlong10_to_str(nr,buff,unsigned_val ? 10 : -10) - buff);
int_part= field_length- (dec ? dec+1 : 0);
Import changeset
2000-07-31 21:29:14 +02:00
if (length > int_part)
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
overflow(!unsigned_val && nr < 0L); /* purecov: inspected */
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
fyllchar = zerofill ? (char) '0' : (char) ' ';
to= ptr;
for (uint i=int_part-length ; i-- > 0 ;)
*to++ = fyllchar;
memcpy(to,buff,length);
if (dec)
Import changeset
2000-07-31 21:29:14 +02:00
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
to[length]='.';
bfill(to+length+1,dec,'0');
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_decimal::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
char *end_not_used;
return my_strntod(&my_charset_bin, ptr, field_length, &end_not_used,
&not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
longlong Field_decimal::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
return my_strntoull(&my_charset_bin, ptr, field_length, 10, NULL,
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
&not_used);
Import changeset
2000-07-31 21:29:14 +02:00
else
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
return my_strntoll(&my_charset_bin, ptr, field_length, 10, NULL,
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
&not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_decimal::val_str(String *val_buffer __attribute__((unused)),
String *val_ptr)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
char *str;
for (str=ptr ; *str == ' ' ; str++) ;
uint tmp_length=(uint) (str-ptr);
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
val_ptr->set_charset(&my_charset_bin);
Import changeset
2000-07-31 21:29:14 +02:00
if (field_length < tmp_length) // Error in data
val_ptr->length(0);
else
MY_CS_NONTEXT flag for sprintf() incompatible charsets (ucs2 if the only one now) Some default_charset_info were removed
2003-06-03 11:59:17 +02:00
val_ptr->set_ascii((const char*) str, field_length-tmp_length);
Import changeset
2000-07-31 21:29:14 +02:00
return val_ptr;
}
/*
** Should be able to handle at least the following fixed decimal formats:
** 5.00 , -1.0, 05, -05, +5 with optional pre/end space
*/
int Field_decimal::cmp(const char *a_ptr,const char *b_ptr)
{
const char *end;
Removed not used variable 'last_ref' Fixed problem with negative DECIMAL() keys Fixed some bugs with NULL keys in BDB More mysql-test tests
2001-01-16 05:02:05 +01:00
int swap=0;
Import changeset
2000-07-31 21:29:14 +02:00
/* First remove prefixes '0', ' ', and '-' */
for (end=a_ptr+field_length;
a_ptr != end &&
(*a_ptr == *b_ptr ||
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
((my_isspace(&my_charset_bin,*a_ptr) || *a_ptr == '+' ||
New ctype functions/macros to support many charsets at a time
2002-03-12 18:37:58 +01:00
*a_ptr == '0') &&
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
(my_isspace(&my_charset_bin,*b_ptr) || *b_ptr == '+' ||
New ctype functions/macros to support many charsets at a time
2002-03-12 18:37:58 +01:00
*b_ptr == '0')));
Removed not used variable 'last_ref' Fixed problem with negative DECIMAL() keys Fixed some bugs with NULL keys in BDB More mysql-test tests
2001-01-16 05:02:05 +01:00
a_ptr++,b_ptr++)
{
if (*a_ptr == '-') // If both numbers are negative
swap= -1 ^ 1; // Swap result
}
Import changeset
2000-07-31 21:29:14 +02:00
if (a_ptr == end)
return 0;
if (*a_ptr == '-')
Removed not used variable 'last_ref' Fixed problem with negative DECIMAL() keys Fixed some bugs with NULL keys in BDB More mysql-test tests
2001-01-16 05:02:05 +01:00
return -1;
cleanup: if there is return from if-part, we don't need else statement
2003-12-10 20:26:31 +01:00
if (*b_ptr == '-')
Import changeset
2000-07-31 21:29:14 +02:00
return 1;
while (a_ptr != end)
{
if (*a_ptr++ != *b_ptr++)
return swap ^ (a_ptr[-1] < b_ptr[-1] ? -1 : 1); // compare digits
}
return 0;
}
void Field_decimal::sort_string(char *to,uint length)
{
char *str,*end;
for (str=ptr,end=ptr+length;
str != end &&
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
((my_isspace(&my_charset_bin,*str) || *str == '+' ||
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
*str == '0')) ;
Import changeset
2000-07-31 21:29:14 +02:00
str++)
*to++=' ';
if (str == end)
return; /* purecov: inspected */
if (*str == '-')
{
*to++=1; // Smaller than any number
str++;
while (str != end)
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
if (my_isdigit(&my_charset_bin,*str))
Import changeset
2000-07-31 21:29:14 +02:00
*to++= (char) ('9' - *str++);
else
*to++= *str++;
}
else memcpy(to,str,(uint) (end-str));
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
void Field_decimal::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
Import changeset
2000-07-31 21:29:14 +02:00
uint tmp=field_length;
if (!unsigned_flag)
tmp--;
if (dec)
tmp--;
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"decimal(%d,%d)",tmp,dec));
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
/****************************************************************************
** Field_new_decimal
****************************************************************************/
Field_new_decimal::Field_new_decimal(char *ptr_arg,
uint32 len_arg, uchar *null_ptr_arg,
uchar null_bit_arg,
enum utype unireg_check_arg,
const char *field_name_arg,
uint8 dec_arg,bool zero_arg,
bool unsigned_arg)
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
:Field_num(ptr_arg, len_arg, null_ptr_arg, null_bit_arg,
unireg_check_arg, field_name_arg, dec_arg, zero_arg, unsigned_arg)
Precision Math implementation
2005-02-08 23:50:45 +01:00
{
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
precision= my_decimal_length_to_precision(len_arg, dec_arg, unsigned_arg);
DBUG_ASSERT((precision <= DECIMAL_MAX_PRECISION) &&
(dec <= DECIMAL_MAX_SCALE));
bin_size= my_decimal_get_binary_size(precision, dec);
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
Field_new_decimal::Field_new_decimal(uint32 len_arg,
bool maybe_null,
const char *name,
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
uint8 dec_arg,
bool unsigned_arg)
:Field_num((char*) 0, len_arg,
Precision Math implementation
2005-02-08 23:50:45 +01:00
maybe_null ? (uchar*) "": 0, 0,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
NONE, name, dec_arg, 0, unsigned_arg)
Precision Math implementation
2005-02-08 23:50:45 +01:00
{
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
precision= my_decimal_length_to_precision(len_arg, dec_arg, unsigned_arg);
DBUG_ASSERT((precision <= DECIMAL_MAX_PRECISION) &&
(dec <= DECIMAL_MAX_SCALE));
bin_size= my_decimal_get_binary_size(precision, dec);
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
void Field_new_decimal::reset(void)
{
store_value(&decimal_zero);
}
/*
Generate max/min decimal value in case of overflow.
SYNOPSIS
Field_new_decimal::set_value_on_overflow();
decimal_value buffer for value
sign sign of value which caused overflow
*/
void Field_new_decimal::set_value_on_overflow(my_decimal *decimal_value,
bool sign)
{
DBUG_ENTER("Field_new_decimal::set_value_on_overflow");
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
max_my_decimal(decimal_value, precision, decimals());
Precision Math implementation
2005-02-08 23:50:45 +01:00
if (sign)
{
if (unsigned_flag)
my_decimal_set_zero(decimal_value);
else
decimal_value->sign(TRUE);
}
DBUG_VOID_RETURN;
}
/*
Store decimal value in the binary buffer
SYNOPSIS
store_value(const my_decimal *decimal_value)
decimal_value my_decimal
DESCRIPTION
checks if decimal_value fits into field size.
if it does, stores the decimal in the buffer using binary format.
Otherwise sets maximal number that can be stored in the field.
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
RETURN
0 ok
1 error
Precision Math implementation
2005-02-08 23:50:45 +01:00
*/
bool Field_new_decimal::store_value(const my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
int error= 0;
DBUG_ENTER("Field_new_decimal::store_value");
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
#ifndef DBUG_OFF
{
char dbug_buff[DECIMAL_MAX_STR_LENGTH+1];
DBUG_PRINT("enter", ("value: %s", dbug_decimal_as_string(dbug_buff, decimal_value)));
}
#endif
Precision Math implementation
2005-02-08 23:50:45 +01:00
/* check that we do not try to write negative value in unsigned field */
if (unsigned_flag && decimal_value->sign())
{
DBUG_PRINT("info", ("unsigned overflow"));
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
error= 1;
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
decimal_value= &decimal_zero;
}
#ifndef DBUG_OFF
{
char dbug_buff[DECIMAL_MAX_STR_LENGTH+1];
Moved long running query to type_newdecimal-big.test Removed warnings that depends on floating point comparisions in type_newdecimal.test which caused failures in some setups
2005-11-23 19:16:06 +01:00
DBUG_PRINT("info", ("saving with precision %d scale: %d value %s",
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
(int)precision, (int)dec,
dbug_decimal_as_string(dbug_buff, decimal_value)));
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
#endif
Precision Math implementation
2005-02-08 23:50:45 +01:00
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
if (warn_if_overflow(my_decimal2binary(E_DEC_FATAL_ERROR & ~E_DEC_OVERFLOW,
decimal_value, ptr, precision, dec)))
Precision Math implementation
2005-02-08 23:50:45 +01:00
{
my_decimal buff;
DBUG_PRINT("info", ("overflow"));
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
set_value_on_overflow(&buff, decimal_value->sign());
my_decimal2binary(E_DEC_FATAL_ERROR, &buff, ptr, precision, dec);
use dbug_print_decimal instead of DBUG_EXECUTE Portability fix Note: rpl_trunc_binlog fails, but Sergei has promised to fix it, so I will ignore it for now
2005-02-19 23:24:30 +01:00
error= 1;
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
Moved long running query to type_newdecimal-big.test Removed warnings that depends on floating point comparisions in type_newdecimal.test which caused failures in some setups
2005-11-23 19:16:06 +01:00
DBUG_EXECUTE("info", print_decimal_buff(decimal_value, (byte *) ptr,
bin_size););
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
DBUG_RETURN(error);
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
int Field_new_decimal::store(const char *from, uint length,
CHARSET_INFO *charset)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Precision Math implementation
2005-02-08 23:50:45 +01:00
int err;
my_decimal decimal_value;
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
DBUG_ENTER("Field_new_decimal::store(char*)");
a fix (bug #6961: Traditional: assignment of string to numeric column should fail).
2005-05-13 10:22:27 +02:00
if ((err= str2my_decimal(E_DEC_FATAL_ERROR &
~(E_DEC_OVERFLOW | E_DEC_BAD_NUM),
from, length, charset, &decimal_value)) &&
table->in_use->abort_on_warning)
{
push_warning_printf(table->in_use, MYSQL_ERROR::WARN_LEVEL_ERROR,
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD,
ER(ER_TRUNCATED_WRONG_VALUE_FOR_FIELD),
"decimal", from, field_name,
(ulong) table->in_use->row_count);
DBUG_RETURN(err);
}
switch (err) {
Precision Math implementation
2005-02-08 23:50:45 +01:00
case E_DEC_TRUNCATED:
set_warning(MYSQL_ERROR::WARN_LEVEL_NOTE, WARN_DATA_TRUNCATED, 1);
break;
case E_DEC_OVERFLOW:
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
set_value_on_overflow(&decimal_value, decimal_value.sign());
break;
case E_DEC_BAD_NUM:
push_warning_printf(table->in_use, MYSQL_ERROR::WARN_LEVEL_WARN,
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD,
ER(ER_TRUNCATED_WRONG_VALUE_FOR_FIELD),
"decimal", from, field_name,
(ulong) table->in_use->row_count);
my_decimal_set_zero(&decimal_value);
break;
}
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
#ifndef DBUG_OFF
char dbug_buff[DECIMAL_MAX_STR_LENGTH+1];
DBUG_PRINT("enter", ("value: %s",
dbug_decimal_as_string(dbug_buff, &decimal_value)));
#endif
Precision Math implementation
2005-02-08 23:50:45 +01:00
store_value(&decimal_value);
DBUG_RETURN(err);
}
int Field_new_decimal::store(double nr)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal decimal_value;
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
int err;
DBUG_ENTER("Field_new_decimal::store(double)");
err= double2my_decimal(E_DEC_FATAL_ERROR & ~E_DEC_OVERFLOW, nr,
&decimal_value);
Precision Math implementation
2005-02-08 23:50:45 +01:00
/*
TODO: fix following when double2my_decimal when double2decimal
will return E_DEC_TRUNCATED always correctly
*/
if (!err)
{
double nr2;
my_decimal2double(E_DEC_FATAL_ERROR, &decimal_value, &nr2);
if (nr2 != nr)
err= E_DEC_TRUNCATED;
}
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
if (err)
{
if (check_overflow(err))
set_value_on_overflow(&decimal_value, decimal_value.sign());
/* Only issue a warning if store_value doesn't issue an warning */
table->in_use->got_warning= 0;
}
if (store_value(&decimal_value))
err= 1;
else if (err && !table->in_use->got_warning)
err= warn_if_overflow(err);
DBUG_RETURN(err);
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_new_decimal::store(longlong nr, bool unsigned_val)
Precision Math implementation
2005-02-08 23:50:45 +01:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal decimal_value;
int err;
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
if ((err= int2my_decimal(E_DEC_FATAL_ERROR & ~E_DEC_OVERFLOW,
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
nr, unsigned_val, &decimal_value)))
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
{
if (check_overflow(err))
set_value_on_overflow(&decimal_value, decimal_value.sign());
/* Only issue a warning if store_value doesn't issue an warning */
table->in_use->got_warning= 0;
}
if (store_value(&decimal_value))
err= 1;
else if (err && !table->in_use->got_warning)
err= warn_if_overflow(err);
Precision Math implementation
2005-02-08 23:50:45 +01:00
return err;
}
int Field_new_decimal::store_decimal(const my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Precision Math implementation
2005-02-08 23:50:45 +01:00
return store_value(decimal_value);
}
double Field_new_decimal::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
double dbl;
my_decimal decimal_value;
my_decimal2double(E_DEC_FATAL_ERROR, val_decimal(&decimal_value), &dbl);
return dbl;
}
longlong Field_new_decimal::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
longlong i;
my_decimal decimal_value;
my_decimal2int(E_DEC_FATAL_ERROR, val_decimal(&decimal_value),
unsigned_flag, &i);
return i;
}
my_decimal* Field_new_decimal::val_decimal(my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
DBUG_ENTER("Field_new_decimal::val_decimal");
binary2my_decimal(E_DEC_FATAL_ERROR, ptr, decimal_value,
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
precision, dec);
use dbug_print_decimal instead of DBUG_EXECUTE Portability fix Note: rpl_trunc_binlog fails, but Sergei has promised to fix it, so I will ignore it for now
2005-02-19 23:24:30 +01:00
DBUG_EXECUTE("info", print_decimal_buff(decimal_value, (byte *) ptr,
bin_size););
Precision Math implementation
2005-02-08 23:50:45 +01:00
DBUG_RETURN(decimal_value);
}
String *Field_new_decimal::val_str(String *val_buffer,
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
String *val_ptr __attribute__((unused)))
Precision Math implementation
2005-02-08 23:50:45 +01:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal decimal_value;
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
uint fixed_precision= zerofill ? precision : 0;
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal2string(E_DEC_FATAL_ERROR, val_decimal(&decimal_value),
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
fixed_precision, dec, '0', val_buffer);
Precision Math implementation
2005-02-08 23:50:45 +01:00
return val_buffer;
}
int Field_new_decimal::cmp(const char *a,const char*b)
{
return memcmp(a, b, bin_size);
}
void Field_new_decimal::sort_string(char *buff,
uint length __attribute__((unused)))
{
memcpy(buff, ptr, bin_size);
}
void Field_new_decimal::sql_type(String &str) const
{
CHARSET_INFO *cs= str.charset();
str.length(cs->cset->snprintf(cs, (char*) str.ptr(), str.alloced_length(),
A lot of fixes to Precision math Mostly about precision/decimals of the results of the operations
2005-05-05 17:06:49 +02:00
"decimal(%d,%d)", precision, (int)dec));
Precision Math implementation
2005-02-08 23:50:45 +01:00
add_zerofill_and_unsigned(str);
}
Things missing in last changset After merge fixes Don't give duplicate warnings for some ::store() functions
2005-02-10 14:41:51 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
** tiny int
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_tiny::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used; // We can ignore result from str2int
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
char *end;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
long tmp= my_strntol(cs, from, len, 10, &end, &not_used);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
if (tmp < 0)
{
tmp=0; /* purecov: inspected */
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (tmp > 255)
{
tmp= 255;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (table->in_use->count_cuted_fields && check_int(from,len,end,cs))
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
if (tmp < -128)
{
tmp= -128;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (tmp >= 128)
{
tmp= 127;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (table->in_use->count_cuted_fields && check_int(from,len,end,cs))
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
ptr[0]= (char) tmp;
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_tiny::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
nr=rint(nr);
if (unsigned_flag)
{
if (nr < 0.0)
{
*ptr=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr > 255.0)
{
*ptr=(char) 255;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
*ptr=(char) nr;
}
else
{
if (nr < -128.0)
{
*ptr= (char) -128;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr > 127.0)
{
*ptr=127;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
compatibility fixes
2004-11-29 22:47:50 +01:00
*ptr=(char) (int) nr;
Import changeset
2000-07-31 21:29:14 +02:00
}
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_tiny::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0 && !unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
*ptr= 0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
else if ((ulonglong) nr > (ulonglong) 255)
Import changeset
2000-07-31 21:29:14 +02:00
{
*ptr= (char) 255;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
*ptr=(char) nr;
}
else
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0 && unsigned_val)
nr= 256; // Generate overflow
if (nr < -128)
Import changeset
2000-07-31 21:29:14 +02:00
{
*ptr= (char) -128;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
else if (nr > 127)
Import changeset
2000-07-31 21:29:14 +02:00
{
*ptr=127;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
*ptr=(char) nr;
}
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_tiny::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
int tmp= unsigned_flag ? (int) ((uchar*) ptr)[0] :
(int) ((signed char*) ptr)[0];
return (double) tmp;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
longlong Field_tiny::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
int tmp= unsigned_flag ? (int) ((uchar*) ptr)[0] :
(int) ((signed char*) ptr)[0];
return (longlong) tmp;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_tiny::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
CHARSET_INFO *cs= &my_charset_bin;
Import changeset
2000-07-31 21:29:14 +02:00
uint length;
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
uint mlength=max(field_length+1,5*cs->mbmaxlen);
val_buffer->alloc(mlength);
Import changeset
2000-07-31 21:29:14 +02:00
char *to=(char*) val_buffer->ptr();
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length= (uint) cs->cset->long10_to_str(cs,to,mlength, 10,
(long) *((uchar*) ptr));
Import changeset
2000-07-31 21:29:14 +02:00
else
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length= (uint) cs->cset->long10_to_str(cs,to,mlength,-10,
(long) *((signed char*) ptr));
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->length(length);
if (zerofill)
prepend_zeros(val_buffer);
return val_buffer;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_tiny::send_binary(Protocol *protocol)
{
return protocol->store_tiny((longlong) (int8) ptr[0]);
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_tiny::cmp(const char *a_ptr, const char *b_ptr)
{
signed char a,b;
a=(signed char) a_ptr[0]; b= (signed char) b_ptr[0];
if (unsigned_flag)
return ((uchar) a < (uchar) b) ? -1 : ((uchar) a > (uchar) b) ? 1 : 0;
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
void Field_tiny::sort_string(char *to,uint length __attribute__((unused)))
{
if (unsigned_flag)
*to= *ptr;
else
to[0] = (char) ((uchar) ptr[0] ^ (uchar) 128); /* Revers signbit */
}
void Field_tiny::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"tinyint(%d)",(int) field_length));
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
/****************************************************************************
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Field type short int (2 byte)
Import changeset
2000-07-31 21:29:14 +02:00
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_short::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used; // We can ignore result from str2int
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
char *end;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
long tmp= my_strntol(cs, from, len, 10, &end, &not_used);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
if (tmp < 0)
{
tmp=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
compatibility fixes
2004-11-29 22:47:50 +01:00
else if (tmp > UINT_MAX16)
Import changeset
2000-07-31 21:29:14 +02:00
{
compatibility fixes
2004-11-29 22:47:50 +01:00
tmp=UINT_MAX16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (table->in_use->count_cuted_fields && check_int(from,len,end,cs))
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
if (tmp < INT_MIN16)
{
tmp= INT_MIN16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (tmp > INT_MAX16)
{
tmp=INT_MAX16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (table->in_use->count_cuted_fields && check_int(from,len,end,cs))
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int2store(ptr,tmp);
}
else
#endif
shortstore(ptr,(short) tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_short::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
int16 res;
nr=rint(nr);
if (unsigned_flag)
{
if (nr < 0)
{
res=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
compatibility fixes
2004-11-29 22:47:50 +01:00
else if (nr > (double) UINT_MAX16)
Import changeset
2000-07-31 21:29:14 +02:00
{
compatibility fixes
2004-11-29 22:47:50 +01:00
res=(int16) UINT_MAX16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(int16) (uint16) nr;
}
else
{
if (nr < (double) INT_MIN16)
{
res=INT_MIN16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr > (double) INT_MAX16)
{
res=INT_MAX16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
compatibility fixes
2004-11-29 22:47:50 +01:00
res=(int16) (int) nr;
Import changeset
2000-07-31 21:29:14 +02:00
}
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int2store(ptr,res);
}
else
#endif
shortstore(ptr,res);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_short::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
int16 res;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0L && !unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
res=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
else if ((ulonglong) nr > (ulonglong) UINT_MAX16)
Import changeset
2000-07-31 21:29:14 +02:00
{
compatibility fixes
2004-11-29 22:47:50 +01:00
res=(int16) UINT_MAX16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(int16) (uint16) nr;
}
else
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0 && unsigned_val)
nr= UINT_MAX16+1; // Generate overflow
Import changeset
2000-07-31 21:29:14 +02:00
if (nr < INT_MIN16)
{
res=INT_MIN16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
else if (nr > (longlong) INT_MAX16)
Import changeset
2000-07-31 21:29:14 +02:00
{
res=INT_MAX16;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(int16) nr;
}
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int2store(ptr,res);
}
else
#endif
shortstore(ptr,res);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_short::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
short j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint2korr(ptr);
else
#endif
shortget(j,ptr);
return unsigned_flag ? (double) (unsigned short) j : (double) j;
}
longlong Field_short::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
short j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint2korr(ptr);
else
#endif
shortget(j,ptr);
return unsigned_flag ? (longlong) (unsigned short) j : (longlong) j;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_short::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
CHARSET_INFO *cs= &my_charset_bin;
Import changeset
2000-07-31 21:29:14 +02:00
uint length;
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
uint mlength=max(field_length+1,7*cs->mbmaxlen);
val_buffer->alloc(mlength);
Import changeset
2000-07-31 21:29:14 +02:00
char *to=(char*) val_buffer->ptr();
short j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint2korr(ptr);
else
#endif
shortget(j,ptr);
if (unsigned_flag)
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length=(uint) cs->cset->long10_to_str(cs, to, mlength, 10,
(long) (uint16) j);
Import changeset
2000-07-31 21:29:14 +02:00
else
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length=(uint) cs->cset->long10_to_str(cs, to, mlength,-10, (long) j);
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->length(length);
if (zerofill)
prepend_zeros(val_buffer);
return val_buffer;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_short::send_binary(Protocol *protocol)
{
return protocol->store_short(Field_short::val_int());
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_short::cmp(const char *a_ptr, const char *b_ptr)
{
short a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
a=sint2korr(a_ptr);
b=sint2korr(b_ptr);
}
else
#endif
{
shortget(a,a_ptr);
shortget(b,b_ptr);
}
if (unsigned_flag)
return ((unsigned short) a < (unsigned short) b) ? -1 :
((unsigned short) a > (unsigned short) b) ? 1 : 0;
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
void Field_short::sort_string(char *to,uint length __attribute__((unused)))
{
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (!table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
if (unsigned_flag)
to[0] = ptr[0];
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
to[0] = (char) (ptr[0] ^ 128); /* Revers signbit */
Import changeset
2000-07-31 21:29:14 +02:00
to[1] = ptr[1];
}
else
#endif
{
if (unsigned_flag)
to[0] = ptr[1];
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
to[0] = (char) (ptr[1] ^ 128); /* Revers signbit */
Import changeset
2000-07-31 21:29:14 +02:00
to[1] = ptr[0];
}
}
void Field_short::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"smallint(%d)",(int) field_length));
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
/****************************************************************************
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Field type medium int (3 byte)
Import changeset
2000-07-31 21:29:14 +02:00
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_medium::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used; // We can ignore result from str2int
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
char *end;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
long tmp= my_strntol(cs, from, len, 10, &end, &not_used);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
if (tmp < 0)
{
tmp=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (tmp >= (long) (1L << 24))
{
tmp=(long) (1L << 24)-1L;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (table->in_use->count_cuted_fields && check_int(from,len,end,cs))
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
if (tmp < INT_MIN24)
{
tmp= INT_MIN24;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (tmp > INT_MAX24)
{
tmp=INT_MAX24;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (table->in_use->count_cuted_fields && check_int(from,len,end,cs))
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
int3store(ptr,tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_medium::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
nr=rint(nr);
if (unsigned_flag)
{
if (nr < 0)
{
int3store(ptr,0);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr >= (double) (long) (1L << 24))
{
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 tmp=(uint32) (1L << 24)-1L;
Import changeset
2000-07-31 21:29:14 +02:00
int3store(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
int3store(ptr,(uint32) nr);
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
if (nr < (double) INT_MIN24)
{
long tmp=(long) INT_MIN24;
int3store(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr > (double) INT_MAX24)
{
long tmp=(long) INT_MAX24;
int3store(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
int3store(ptr,(long) nr);
}
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_medium::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0 && !unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
int3store(ptr,0);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
else if ((ulonglong) nr >= (ulonglong) (long) (1L << 24))
Import changeset
2000-07-31 21:29:14 +02:00
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
long tmp= (long) (1L << 24)-1L;
Import changeset
2000-07-31 21:29:14 +02:00
int3store(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
int3store(ptr,(uint32) nr);
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0 && unsigned_val)
nr= (ulonglong) (long) (1L << 24); // Generate overflow
Import changeset
2000-07-31 21:29:14 +02:00
if (nr < (longlong) INT_MIN24)
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
long tmp= (long) INT_MIN24;
Import changeset
2000-07-31 21:29:14 +02:00
int3store(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr > (longlong) INT_MAX24)
{
long tmp=(long) INT_MAX24;
int3store(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
int3store(ptr,(long) nr);
}
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_medium::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
long j= unsigned_flag ? (long) uint3korr(ptr) : sint3korr(ptr);
return (double) j;
}
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
longlong Field_medium::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
long j= unsigned_flag ? (long) uint3korr(ptr) : sint3korr(ptr);
return (longlong) j;
}
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_medium::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
CHARSET_INFO *cs= &my_charset_bin;
Import changeset
2000-07-31 21:29:14 +02:00
uint length;
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
uint mlength=max(field_length+1,10*cs->mbmaxlen);
val_buffer->alloc(mlength);
Import changeset
2000-07-31 21:29:14 +02:00
char *to=(char*) val_buffer->ptr();
long j= unsigned_flag ? (long) uint3korr(ptr) : sint3korr(ptr);
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length=(uint) cs->cset->long10_to_str(cs,to,mlength,-10,j);
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->length(length);
if (zerofill)
prepend_zeros(val_buffer); /* purecov: inspected */
return val_buffer;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_medium::send_binary(Protocol *protocol)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
return protocol->store_long(Field_medium::val_int());
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_medium::cmp(const char *a_ptr, const char *b_ptr)
{
long a,b;
if (unsigned_flag)
{
a=uint3korr(a_ptr);
b=uint3korr(b_ptr);
}
else
{
a=sint3korr(a_ptr);
b=sint3korr(b_ptr);
}
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
void Field_medium::sort_string(char *to,uint length __attribute__((unused)))
{
if (unsigned_flag)
to[0] = ptr[2];
else
to[0] = (uchar) (ptr[2] ^ 128); /* Revers signbit */
to[1] = ptr[1];
to[2] = ptr[0];
}
void Field_medium::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"mediumint(%d)",(int) field_length));
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
/****************************************************************************
** long int
****************************************************************************/
field.cc: optimize test_if_minus() when not UCS2 support is compiled.
2005-02-22 05:35:15 +01:00
/*
A helper function to check whether the next character
in the string "s" is MINUS SIGN.
*/
#ifdef HAVE_CHARSET_ucs2
Bug#8235 Connection collation change & table create with default result in crash
2005-02-12 13:27:22 +01:00
static bool test_if_minus(CHARSET_INFO *cs,
const char *s, const char *e)
{
my_wc_t wc;
return cs->cset->mb_wc(cs, &wc, (uchar*) s, (uchar*) e) > 0 && wc == '-';
}
field.cc: optimize test_if_minus() when not UCS2 support is compiled.
2005-02-22 05:35:15 +01:00
#else
/*
If not UCS2 support is compiled then it is easier
*/
#define test_if_minus(cs, s, e) (*s == '-')
#endif
Bug#8235 Connection collation change & table create with default result in crash
2005-02-12 13:27:22 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_long::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Eliminate warnings noticed by VC7. This includes fixing my_mmap() on Windows to call CreateFileMapping() with correct arguments, and propogating the introduction of query_id_t to everywhere query ids are passed around. (Bug #8826)
2005-03-19 01:12:25 +01:00
ulong tmp_scan;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
longlong tmp;
long store_tmp;
int error;
Removed copying of parameters as this leads to memory leaks in embedded server. Fixed 'not initialized' memory error.
2002-12-05 02:40:33 +01:00
char *end;
XID SQL syntax minor cleanups XA tests
2005-04-04 00:50:05 +02:00
Eliminate warnings noticed by VC7. This includes fixing my_mmap() on Windows to call CreateFileMapping() with correct arguments, and propogating the introduction of query_id_t to everywhere query ids are passed around. (Bug #8826)
2005-03-19 01:12:25 +01:00
tmp_scan= cs->cset->scan(cs, from, from+len, MY_SEQ_SPACES);
len-= tmp_scan;
from+= tmp_scan;
manually merged
2004-08-18 19:57:55 +02:00
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
end= (char*) from+len;
XID SQL syntax minor cleanups XA tests
2005-04-04 00:50:05 +02:00
tmp= cs->cset->strtoll10(cs, from, &end, &error);
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error != MY_ERRNO_EDOM)
Import changeset
2000-07-31 21:29:14 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (unsigned_flag)
Import changeset
2000-07-31 21:29:14 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error < 0)
{
error= 1;
tmp= 0;
}
else if ((ulonglong) tmp > (ulonglong) UINT_MAX32)
{
tmp= UINT_MAX32;
error= 1;
}
else
error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
Bug #4797 - 32 bit and 64 bit builds behave differently on int32 overflow
2004-08-11 23:43:41 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error < 0)
{
error= 0;
if (tmp < INT_MIN32)
{
tmp= INT_MIN32;
error= 1;
}
}
else if (tmp > INT_MAX32)
{
tmp= INT_MAX32;
error= 1;
}
Bug #4797 - 32 bit and 64 bit builds behave differently on int32 overflow
2004-08-11 23:43:41 +02:00
}
}
manually merged
2004-08-18 19:57:55 +02:00
if (error)
After merge fixes Added THD to add_to_xxx_list() functions for faster parsing.
2002-12-06 20:11:27 +01:00
{
field.cc: Correction after manula merge.
2005-06-24 10:21:18 +02:00
error= error != MY_ERRNO_EDOM ? 1 : 2;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (from+len != end && table->in_use->count_cuted_fields &&
check_int(from,len,end,cs))
Manual merge
2005-06-23 20:22:30 +02:00
error= 2;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
store_tmp= (long) tmp;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
int4store(ptr, store_tmp);
Import changeset
2000-07-31 21:29:14 +02:00
}
else
#endif
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
longstore(ptr, store_tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_long::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
int32 res;
nr=rint(nr);
if (unsigned_flag)
{
if (nr < 0)
{
res=0;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
A fix. Why didn't we use UINT_MAX32?
2004-11-24 19:18:28 +01:00
else if (nr > (double) UINT_MAX32)
Import changeset
2000-07-31 21:29:14 +02:00
{
A fix. Why didn't we use UINT_MAX32?
2004-11-24 19:18:28 +01:00
res= UINT_MAX32;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(int32) (ulong) nr;
}
else
{
if (nr < (double) INT_MIN32)
{
res=(int32) INT_MIN32;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr > (double) INT_MAX32)
{
res=(int32) INT_MAX32;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
compatibility fixes
2004-11-29 22:47:50 +01:00
res=(int32) (longlong) nr;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error)
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,res);
}
else
#endif
longstore(ptr,res);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_long::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
int32 res;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
DBUG_ASSERT(table->in_use == current_thd); // General safety
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0 && !unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
res=0;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
else if ((ulonglong) nr >= (LL(1) << 32))
Import changeset
2000-07-31 21:29:14 +02:00
{
res=(int32) (uint32) ~0L;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(int32) (uint32) nr;
}
else
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < 0 && unsigned_val)
Some minor fixes revealed as warnings by Intel compiler.
2005-09-23 20:28:56 +02:00
nr= ((longlong) INT_MAX32) + 1; // Generate overflow
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < (longlong) INT_MIN32)
Import changeset
2000-07-31 21:29:14 +02:00
{
res=(int32) INT_MIN32;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr > (longlong) INT_MAX32)
{
res=(int32) INT_MAX32;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(int32) nr;
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error)
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,res);
}
else
#endif
longstore(ptr,res);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_long::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
int32 j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint4korr(ptr);
else
#endif
longget(j,ptr);
return unsigned_flag ? (double) (uint32) j : (double) j;
}
longlong Field_long::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
int32 j;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
/* See the comment in Field_long::store(long long) */
DBUG_ASSERT(table->in_use == current_thd);
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint4korr(ptr);
else
#endif
longget(j,ptr);
return unsigned_flag ? (longlong) (uint32) j : (longlong) j;
}
String *Field_long::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
CHARSET_INFO *cs= &my_charset_bin;
Import changeset
2000-07-31 21:29:14 +02:00
uint length;
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
uint mlength=max(field_length+1,12*cs->mbmaxlen);
val_buffer->alloc(mlength);
Import changeset
2000-07-31 21:29:14 +02:00
char *to=(char*) val_buffer->ptr();
int32 j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint4korr(ptr);
else
#endif
longget(j,ptr);
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
if (unsigned_flag)
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length=cs->cset->long10_to_str(cs,to,mlength, 10,(long) (uint32)j);
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
else
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length=cs->cset->long10_to_str(cs,to,mlength,-10,(long) j);
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->length(length);
if (zerofill)
prepend_zeros(val_buffer);
return val_buffer;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_long::send_binary(Protocol *protocol)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
return protocol->store_long(Field_long::val_int());
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_long::cmp(const char *a_ptr, const char *b_ptr)
{
int32 a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
a=sint4korr(a_ptr);
b=sint4korr(b_ptr);
}
else
#endif
{
longget(a,a_ptr);
longget(b,b_ptr);
}
if (unsigned_flag)
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
return ((uint32) a < (uint32) b) ? -1 : ((uint32) a > (uint32) b) ? 1 : 0;
Import changeset
2000-07-31 21:29:14 +02:00
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
void Field_long::sort_string(char *to,uint length __attribute__((unused)))
{
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (!table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
if (unsigned_flag)
to[0] = ptr[0];
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
to[0] = (char) (ptr[0] ^ 128); /* Revers signbit */
Import changeset
2000-07-31 21:29:14 +02:00
to[1] = ptr[1];
to[2] = ptr[2];
to[3] = ptr[3];
}
else
#endif
{
if (unsigned_flag)
to[0] = ptr[3];
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
to[0] = (char) (ptr[3] ^ 128); /* Revers signbit */
Import changeset
2000-07-31 21:29:14 +02:00
to[1] = ptr[2];
to[2] = ptr[1];
to[3] = ptr[0];
}
}
void Field_long::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"int(%d)",(int) field_length));
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
/****************************************************************************
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
Field type longlong int (8 bytes)
Import changeset
2000-07-31 21:29:14 +02:00
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_longlong::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
longlong tmp;
int error= 0;
Removed copying of parameters as this leads to memory leaks in embedded server. Fixed 'not initialized' memory error.
2002-12-05 02:40:33 +01:00
char *end;
Fix warnings and compilation failures (Windows).
2004-11-17 11:45:05 +01:00
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
tmp= cs->cset->scan(cs, from, from+len, MY_SEQ_SPACES);
Windows compile fixups (To be verified by respective devs)
2003-02-05 03:50:48 +01:00
len-= (uint)tmp;
field.cc: Use cs->scan() instead of my_isspace()
2003-01-27 15:46:51 +01:00
from+= tmp;
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
Bug#8235 Connection collation change & table create with default result in crash
2005-02-12 13:27:22 +01:00
if (!len || test_if_minus(cs, from, from + len))
Import changeset
2000-07-31 21:29:14 +02:00
{
tmp=0; // Set negative to 0
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
tmp=(longlong) my_strntoull(cs,from,len,10,&end,&error);
Import changeset
2000-07-31 21:29:14 +02:00
}
else
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
tmp=my_strntoll(cs,from,len,10,&end,&error);
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error)
Fix to return appropriate warnings using SHOW WARNINGS for LOAD DATA INFILE (scrum)
2003-04-22 09:53:07 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Fix to return appropriate warnings using SHOW WARNINGS for LOAD DATA INFILE (scrum)
2003-04-22 09:53:07 +02:00
error= 1;
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
else if (from+len != end && table->in_use->count_cuted_fields &&
check_int(from,len,end,cs))
Manual merge
2005-06-23 20:22:30 +02:00
error= 2;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int8store(ptr,tmp);
}
else
#endif
longlongstore(ptr,tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_longlong::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
longlong res;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
nr= rint(nr);
Import changeset
2000-07-31 21:29:14 +02:00
if (unsigned_flag)
{
if (nr < 0)
{
res=0;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
compatibility fixes
2004-11-29 22:47:50 +01:00
else if (nr >= (double) ULONGLONG_MAX)
Import changeset
2000-07-31 21:29:14 +02:00
{
res= ~(longlong) 0;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(longlong) (ulonglong) nr;
}
else
{
compatibility fixes
2004-11-29 22:47:50 +01:00
if (nr <= (double) LONGLONG_MIN)
Import changeset
2000-07-31 21:29:14 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
res= LONGLONG_MIN;
compatibility fixes
2004-11-29 22:47:50 +01:00
error= (nr < (double) LONGLONG_MIN);
Import changeset
2000-07-31 21:29:14 +02:00
}
Fix for bug #6077: Problem with make test from 4.1.7 and bk portability fix(UnixWare)
2004-11-10 16:16:58 +01:00
else if (nr >= (double) (ulonglong) LONGLONG_MAX)
Import changeset
2000-07-31 21:29:14 +02:00
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
res= LONGLONG_MAX;
compatibility fixes
2004-11-29 22:47:50 +01:00
error= (nr > (double) LONGLONG_MAX);
Import changeset
2000-07-31 21:29:14 +02:00
}
else
res=(longlong) nr;
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error)
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int8store(ptr,res);
}
else
#endif
longlongstore(ptr,res);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_longlong::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int error= 0;
if (nr < 0) // Only possible error
{
/*
if field is unsigned and value is signed (< 0) or
if field is signed and value is unsigned we have an overflow
*/
if (unsigned_flag != unsigned_val)
{
nr= unsigned_flag ? (ulonglong) 0 : (ulonglong) LONGLONG_MAX;
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
error= 1;
}
}
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int8store(ptr,nr);
}
else
#endif
longlongstore(ptr,nr);
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_longlong::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
longlong j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
j=sint8korr(ptr);
}
else
#endif
longlongget(j,ptr);
Move test that uses many tables (in query_cache.test) to separate test so that we can get it 'skipped' instead of 'failed' on system where we can't open many files.
2003-08-22 03:07:40 +02:00
/* The following is open coded to avoid a bug in gcc 3.3 */
if (unsigned_flag)
{
ulonglong tmp= (ulonglong) j;
return ulonglong2double(tmp);
}
return (double) j;
Import changeset
2000-07-31 21:29:14 +02:00
}
Move test that uses many tables (in query_cache.test) to separate test so that we can get it 'skipped' instead of 'failed' on system where we can't open many files.
2003-08-22 03:07:40 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
longlong Field_longlong::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
longlong j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint8korr(ptr);
else
#endif
longlongget(j,ptr);
return j;
}
String *Field_longlong::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
CHARSET_INFO *cs= &my_charset_bin;
Import changeset
2000-07-31 21:29:14 +02:00
uint length;
Use of charset dependant num->string conversion routines
2002-12-11 13:04:46 +01:00
uint mlength=max(field_length+1,22*cs->mbmaxlen);
val_buffer->alloc(mlength);
Import changeset
2000-07-31 21:29:14 +02:00
char *to=(char*) val_buffer->ptr();
longlong j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint8korr(ptr);
else
#endif
longlongget(j,ptr);
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
length=(uint) (cs->cset->longlong10_to_str)(cs,to,mlength,
Portability fixes (for windows) Some changes to the prepared statement protocol to make it easier to use and faster.
2003-01-21 20:07:59 +01:00
unsigned_flag ? 10 : -10, j);
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->length(length);
if (zerofill)
prepend_zeros(val_buffer);
return val_buffer;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_longlong::send_binary(Protocol *protocol)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
return protocol->store_longlong(Field_longlong::val_int(), unsigned_flag);
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_longlong::cmp(const char *a_ptr, const char *b_ptr)
{
longlong a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
a=sint8korr(a_ptr);
b=sint8korr(b_ptr);
}
else
#endif
{
longlongget(a,a_ptr);
longlongget(b,b_ptr);
}
if (unsigned_flag)
return ((ulonglong) a < (ulonglong) b) ? -1 :
((ulonglong) a > (ulonglong) b) ? 1 : 0;
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
void Field_longlong::sort_string(char *to,uint length __attribute__((unused)))
{
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (!table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
if (unsigned_flag)
to[0] = ptr[0];
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
to[0] = (char) (ptr[0] ^ 128); /* Revers signbit */
Import changeset
2000-07-31 21:29:14 +02:00
to[1] = ptr[1];
to[2] = ptr[2];
to[3] = ptr[3];
to[4] = ptr[4];
to[5] = ptr[5];
to[6] = ptr[6];
to[7] = ptr[7];
}
else
#endif
{
if (unsigned_flag)
to[0] = ptr[7];
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
to[0] = (char) (ptr[7] ^ 128); /* Revers signbit */
Import changeset
2000-07-31 21:29:14 +02:00
to[1] = ptr[6];
to[2] = ptr[5];
to[3] = ptr[4];
to[4] = ptr[3];
to[5] = ptr[2];
to[6] = ptr[1];
to[7] = ptr[0];
}
}
void Field_longlong::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"bigint(%d)",(int) field_length));
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
after merge fixes
2004-03-17 23:09:13 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
after merge fixes
2004-03-17 23:09:13 +01:00
single precision float
Import changeset
2000-07-31 21:29:14 +02:00
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_float::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
after merge fixes
2004-03-17 23:09:13 +01:00
int error;
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
char *end;
after merge fixes
2004-03-17 23:09:13 +01:00
double nr= my_strntod(cs,(char*) from,len,&end,&error);
More fixes for strict mode: More tests. Better error messages. Fixed bug when checking if we updated all needed columns for INSERT. Give an error if we encounter a wrong float value during parsing. Don't print DEFAULT for columns without a default value in SHOW CREATE/SHOW FIELDS. Fixed UPDATE IGNORE when using STRICT mode.
2004-10-02 21:20:08 +02:00
if (error || (!len || (uint) (end-from) != len &&
table->in_use->count_cuted_fields))
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
{
More fixes for strict mode: More tests. Better error messages. Fixed bug when checking if we updated all needed columns for INSERT. Give an error if we encounter a wrong float value during parsing. Don't print DEFAULT for columns without a default value in SHOW CREATE/SHOW FIELDS. Fixed UPDATE IGNORE when using STRICT mode.
2004-10-02 21:20:08 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
(error ? ER_WARN_DATA_OUT_OF_RANGE : WARN_DATA_TRUNCATED), 1);
Manual merge
2005-06-23 20:22:30 +02:00
error= error ? 1 : 2;
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
}
after merge fixes
2004-03-17 23:09:13 +01:00
Field_float::store(nr);
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_float::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
float j;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
- added commands --query_vertical and --query_horisontal to client/mysqltest.cc - get my_strtod to return inf - get Field_float::store(double) and Field_double::store(float) to set null for nan value (as extra serg's recomendations to fix for patch on Bug #2082 'mysqldump converts "inf" to null')
2004-03-06 00:00:21 +01:00
if (isnan(nr))
support for unsigned FLOAT/DOUBLE
2002-03-26 23:50:54 +01:00
{
- added commands --query_vertical and --query_horisontal to client/mysqltest.cc - get my_strtod to return inf - get Field_float::store(double) and Field_double::store(float) to set null for nan value (as extra serg's recomendations to fix for patch on Bug #2082 'mysqldump converts "inf" to null')
2004-03-06 00:00:21 +01:00
j= 0;
set_null();
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
support for unsigned FLOAT/DOUBLE
2002-03-26 23:50:54 +01:00
}
- added commands --query_vertical and --query_horisontal to client/mysqltest.cc - get my_strtod to return inf - get Field_float::store(double) and Field_double::store(float) to set null for nan value (as extra serg's recomendations to fix for patch on Bug #2082 'mysqldump converts "inf" to null')
2004-03-06 00:00:21 +01:00
else if (unsigned_flag && nr < 0)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
j= 0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
else
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
double max_value;
if (dec >= NOT_FIXED_DEC)
{
max_value= FLT_MAX;
}
else
{
out-of-bound array access fixed
2004-05-11 12:39:00 +02:00
uint tmp=min(field_length,array_elements(log_10)-1);
max_value= (log_10[tmp]-1)/log_10[dec];
Fixed problems noticed with last build
2004-04-09 06:12:41 +02:00
/*
The following comparison is needed to not get an overflow if nr
is close to FLT_MAX
*/
if (fabs(nr) < FLT_MAX/10.0e+32)
nr= floor(nr*log_10[dec]+0.5)/log_10[dec];
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
}
if (nr < -max_value)
{
j= (float)-max_value;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
error= 1;
}
else if (nr > max_value)
{
j= (float)max_value;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
error= 1;
}
else
j= (float) nr;
Import changeset
2000-07-31 21:29:14 +02:00
}
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float4store(ptr,j);
}
else
#endif
memcpy_fixed(ptr,(byte*) &j,sizeof(j));
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_float::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
return Field_float::store(unsigned_val ? ulonglong2double((ulonglong) nr) :
(double) nr);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
double Field_float::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
float j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float4get(j,ptr);
}
else
#endif
memcpy_fixed((byte*) &j,ptr,sizeof(j));
return ((double) j);
}
longlong Field_float::val_int(void)
{
float j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float4get(j,ptr);
}
else
#endif
memcpy_fixed((byte*) &j,ptr,sizeof(j));
Addition to fix for bug #12956: cast make differ rounding. - use rint() in some other val_int() methods as well.
2005-11-29 15:06:58 +01:00
return (longlong) rint(j);
Import changeset
2000-07-31 21:29:14 +02:00
}
String *Field_float::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
float nr;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float4get(nr,ptr);
}
else
#endif
memcpy_fixed((byte*) &nr,ptr,sizeof(nr));
The environment variable PAGER is now used as the default pager Fixed bug with float columns
2001-04-17 00:45:30 +02:00
uint to_length=max(field_length,70);
val_buffer->alloc(to_length);
Import changeset
2000-07-31 21:29:14 +02:00
char *to=(char*) val_buffer->ptr();
if (dec >= NOT_FIXED_DEC)
{
sprintf(to,"%-*.*g",(int) field_length,FLT_DIG,nr);
to=strcend(to,' ');
*to=0;
}
else
{
#ifdef HAVE_FCONVERT
char buff[70],*pos=buff;
int decpt,sign,tmp_dec=dec;
VOID(sfconvert(&nr,tmp_dec,&decpt,&sign,buff));
if (sign)
{
*to++='-';
}
if (decpt < 0)
{ /* val_buffer is < 0 */
*to++='0';
if (!tmp_dec)
goto end;
*to++='.';
if (-decpt > tmp_dec)
decpt= - (int) tmp_dec;
tmp_dec=(uint) ((int) tmp_dec+decpt);
while (decpt++ < 0)
*to++='0';
}
else if (decpt == 0)
{
*to++= '0';
if (!tmp_dec)
goto end;
*to++='.';
}
else
{
while (decpt-- > 0)
*to++= *pos++;
if (!tmp_dec)
goto end;
*to++='.';
}
while (tmp_dec--)
*to++= *pos++;
FLUSH TABLE table_list Fixes for RENAME TABLE Portability fixes
2000-08-21 23:18:32 +02:00
#else
The environment variable PAGER is now used as the default pager Fixed bug with float columns
2001-04-17 00:45:30 +02:00
#ifdef HAVE_SNPRINTF
to[to_length-1]=0; // Safety
snprintf(to,to_length-1,"%.*f",dec,nr);
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
to=strend(to);
Import changeset
2000-07-31 21:29:14 +02:00
#else
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
to+= my_sprintf(to,(to,"%.*f",dec,nr));
FLUSH TABLE table_list Fixes for RENAME TABLE Portability fixes
2000-08-21 23:18:32 +02:00
#endif
Import changeset
2000-07-31 21:29:14 +02:00
#endif
}
#ifdef HAVE_FCONVERT
end:
#endif
val_buffer->length((uint) (to-val_buffer->ptr()));
if (zerofill)
prepend_zeros(val_buffer);
return val_buffer;
}
int Field_float::cmp(const char *a_ptr, const char *b_ptr)
{
float a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float4get(a,a_ptr);
float4get(b,b_ptr);
}
else
#endif
{
memcpy_fixed(&a,a_ptr,sizeof(float));
memcpy_fixed(&b,b_ptr,sizeof(float));
}
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
#define FLT_EXP_DIG (sizeof(float)*8-FLT_MANT_DIG)
void Field_float::sort_string(char *to,uint length __attribute__((unused)))
{
float nr;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float4get(nr,ptr);
}
else
#endif
memcpy_fixed(&nr,ptr,sizeof(float));
uchar *tmp= (uchar*) to;
if (nr == (float) 0.0)
{ /* Change to zero string */
tmp[0]=(uchar) 128;
bzero((char*) tmp+1,sizeof(nr)-1);
}
else
{
#ifdef WORDS_BIGENDIAN
memcpy_fixed(tmp,&nr,sizeof(nr));
#else
tmp[0]= ptr[3]; tmp[1]=ptr[2]; tmp[2]= ptr[1]; tmp[3]=ptr[0];
#endif
if (tmp[0] & 128) /* Negative */
{ /* make complement */
uint i;
for (i=0 ; i < sizeof(nr); i++)
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
tmp[i]= (uchar) (tmp[i] ^ (uchar) 255);
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
ushort exp_part=(((ushort) tmp[0] << 8) | (ushort) tmp[1] |
(ushort) 32768);
exp_part+= (ushort) 1 << (16-1-FLT_EXP_DIG);
tmp[0]= (uchar) (exp_part >> 8);
tmp[1]= (uchar) exp_part;
}
}
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_float::send_binary(Protocol *protocol)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
return protocol->store((float) Field_float::val_real(), dec, (String*) 0);
}
Import changeset
2000-07-31 21:29:14 +02:00
void Field_float::sql_type(String &res) const
{
if (dec == NOT_FIXED_DEC)
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("float"));
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
else
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
{
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
CHARSET_INFO *cs= res.charset();
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"float(%d,%d)",(int) field_length,dec));
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
after merge fixes
2004-03-17 23:09:13 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
after merge fixes
2004-03-17 23:09:13 +01:00
double precision floating point numbers
Import changeset
2000-07-31 21:29:14 +02:00
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_double::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
after merge fixes
2004-03-17 23:09:13 +01:00
int error;
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
char *end;
after merge fixes
2004-03-17 23:09:13 +01:00
double nr= my_strntod(cs,(char*) from, len, &end, &error);
More fixes for strict mode: More tests. Better error messages. Fixed bug when checking if we updated all needed columns for INSERT. Give an error if we encounter a wrong float value during parsing. Don't print DEFAULT for columns without a default value in SHOW CREATE/SHOW FIELDS. Fixed UPDATE IGNORE when using STRICT mode.
2004-10-02 21:20:08 +02:00
if (error || (!len || (uint) (end-from) != len &&
table->in_use->count_cuted_fields))
Import changeset
2000-07-31 21:29:14 +02:00
{
More fixes for strict mode: More tests. Better error messages. Fixed bug when checking if we updated all needed columns for INSERT. Give an error if we encounter a wrong float value during parsing. Don't print DEFAULT for columns without a default value in SHOW CREATE/SHOW FIELDS. Fixed UPDATE IGNORE when using STRICT mode.
2004-10-02 21:20:08 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
(error ? ER_WARN_DATA_OUT_OF_RANGE : WARN_DATA_TRUNCATED), 1);
Manual merge
2005-06-23 20:22:30 +02:00
error= error ? 1 : 2;
Import changeset
2000-07-31 21:29:14 +02:00
}
after merge fixes
2004-03-17 23:09:13 +01:00
Field_double::store(nr);
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_double::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
- added commands --query_vertical and --query_horisontal to client/mysqltest.cc - get my_strtod to return inf - get Field_float::store(double) and Field_double::store(float) to set null for nan value (as extra serg's recomendations to fix for patch on Bug #2082 'mysqldump converts "inf" to null')
2004-03-06 00:00:21 +01:00
if (isnan(nr))
{
nr= 0;
set_null();
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
- added commands --query_vertical and --query_horisontal to client/mysqltest.cc - get my_strtod to return inf - get Field_float::store(double) and Field_double::store(float) to set null for nan value (as extra serg's recomendations to fix for patch on Bug #2082 'mysqldump converts "inf" to null')
2004-03-06 00:00:21 +01:00
error= 1;
}
else if (unsigned_flag && nr < 0)
support for unsigned FLOAT/DOUBLE
2002-03-26 23:50:54 +01:00
{
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
nr= 0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
support for unsigned FLOAT/DOUBLE
2002-03-26 23:50:54 +01:00
}
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
else
{
double max_value;
if (dec >= NOT_FIXED_DEC)
{
max_value= DBL_MAX;
}
else
{
out-of-bound array access fixed
2004-05-11 12:39:00 +02:00
uint tmp=min(field_length,array_elements(log_10)-1);
max_value= (log_10[tmp]-1)/log_10[dec];
Fixed problems noticed with last build
2004-04-09 06:12:41 +02:00
if (fabs(nr) < DBL_MAX/10.0e+32)
nr= floor(nr*log_10[dec]+0.5)/log_10[dec];
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
}
if (nr < -max_value)
{
nr= -max_value;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
error= 1;
}
else if (nr > max_value)
{
nr= max_value;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Fixed bug #2082 'mysqldump converts "inf" to NULL' and added commands vertical_results and horisontal_results to client/mysqltest.c
2004-03-04 18:08:14 +01:00
error= 1;
}
}
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float8store(ptr,nr);
}
else
#endif
doublestore(ptr,nr);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_double::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
return Field_double::store(unsigned_val ? ulonglong2double((ulonglong) nr) :
(double) nr);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
int Field_real::store_decimal(const my_decimal *dm)
{
double dbl;
my_decimal2double(E_DEC_FATAL_ERROR, dm, &dbl);
return store(dbl);
}
Import changeset
2000-07-31 21:29:14 +02:00
double Field_double::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
double j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float8get(j,ptr);
}
else
#endif
doubleget(j,ptr);
return j;
}
longlong Field_double::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
double j;
Additional 5.0 fix for Bug#15098: CAST(column double TO signed int), wrong result which was fixed originally in 4.1.
2006-03-29 16:31:16 +02:00
longlong res;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float8get(j,ptr);
}
else
#endif
doubleget(j,ptr);
Bug#15098 CAST(column double TO signed int), wrong result field.cc: Adding longlong range checking to return LONGLONG_MIN or LONGLONG_MAX when out of range. Using (longlong) cast only when range is ok. cast.test: Adding test case. cast.result: Fixing results accordingly.
2005-12-06 13:54:13 +01:00
/* Check whether we fit into longlong range */
if (j <= (double) LONGLONG_MIN)
Additional 5.0 fix for Bug#15098: CAST(column double TO signed int), wrong result which was fixed originally in 4.1.
2006-03-29 16:31:16 +02:00
{
res= (longlong) LONGLONG_MIN;
goto warn;
}
Bug#15098 CAST(column double TO signed int), wrong result field.cc: Adding longlong range checking to return LONGLONG_MIN or LONGLONG_MAX when out of range. Using (longlong) cast only when range is ok. cast.test: Adding test case. cast.result: Fixing results accordingly.
2005-12-06 13:54:13 +01:00
if (j >= (double) (ulonglong) LONGLONG_MAX)
Additional 5.0 fix for Bug#15098: CAST(column double TO signed int), wrong result which was fixed originally in 4.1.
2006-03-29 16:31:16 +02:00
{
res= (longlong) LONGLONG_MAX;
goto warn;
}
Addition to fix for bug #12956: cast make differ rounding. - use rint() in some other val_int() methods as well.
2005-11-29 15:06:58 +01:00
return (longlong) rint(j);
Additional 5.0 fix for Bug#15098: CAST(column double TO signed int), wrong result which was fixed originally in 4.1.
2006-03-29 16:31:16 +02:00
warn:
{
char buf[DOUBLE_TO_STRING_CONVERSION_BUFFER_SIZE];
String tmp(buf, sizeof(buf), &my_charset_latin1), *str;
str= val_str(&tmp, 0);
push_warning_printf(current_thd, MYSQL_ERROR::WARN_LEVEL_WARN,
ER_TRUNCATED_WRONG_VALUE,
ER(ER_TRUNCATED_WRONG_VALUE), "INTEGER",
str->c_ptr());
}
return res;
Import changeset
2000-07-31 21:29:14 +02:00
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal *Field_real::val_decimal(my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
double2my_decimal(E_DEC_FATAL_ERROR, val_real(), decimal_value);
return decimal_value;
}
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_double::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
double nr;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float8get(nr,ptr);
}
else
#endif
doubleget(nr,ptr);
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
uint to_length=max(field_length, DOUBLE_TO_STRING_CONVERSION_BUFFER_SIZE);
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->alloc(to_length);
char *to=(char*) val_buffer->ptr();
if (dec >= NOT_FIXED_DEC)
{
sprintf(to,"%-*.*g",(int) field_length,DBL_DIG,nr);
to=strcend(to,' ');
}
else
{
#ifdef HAVE_FCONVERT
field.cc: Serg's typo fix :)
2004-08-19 07:28:40 +02:00
char buff[DOUBLE_TO_STRING_CONVERSION_BUFFER_SIZE];
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
char *pos= buff;
Import changeset
2000-07-31 21:29:14 +02:00
int decpt,sign,tmp_dec=dec;
VOID(fconvert(nr,tmp_dec,&decpt,&sign,buff));
if (sign)
{
*to++='-';
}
if (decpt < 0)
{ /* val_buffer is < 0 */
*to++='0';
if (!tmp_dec)
goto end;
*to++='.';
if (-decpt > tmp_dec)
decpt= - (int) tmp_dec;
tmp_dec=(uint) ((int) tmp_dec+decpt);
while (decpt++ < 0)
*to++='0';
}
else if (decpt == 0)
{
*to++= '0';
if (!tmp_dec)
goto end;
*to++='.';
}
else
{
while (decpt-- > 0)
*to++= *pos++;
if (!tmp_dec)
goto end;
*to++='.';
}
while (tmp_dec--)
*to++= *pos++;
#else
#ifdef HAVE_SNPRINTF
Fix typo
2000-08-22 00:35:50 +02:00
to[to_length-1]=0; // Safety
Fix for snprintf
2000-08-21 23:55:45 +02:00
snprintf(to,to_length-1,"%.*f",dec,nr);
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
to=strend(to);
Import changeset
2000-07-31 21:29:14 +02:00
#else
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
to+= my_sprintf(to,(to,"%.*f",dec,nr));
Import changeset
2000-07-31 21:29:14 +02:00
#endif
#endif
}
#ifdef HAVE_FCONVERT
end:
#endif
val_buffer->length((uint) (to-val_buffer->ptr()));
if (zerofill)
prepend_zeros(val_buffer);
return val_buffer;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_double::send_binary(Protocol *protocol)
{
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
return protocol->store((double) Field_double::val_real(), dec, (String*) 0);
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_double::cmp(const char *a_ptr, const char *b_ptr)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
double a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float8get(a,a_ptr);
float8get(b,b_ptr);
}
else
#endif
{
Use new bitmap interface Patches for Armstrong Removed warnings when using REPAIR TABLE .. EXTENDED
2001-02-07 16:42:20 +01:00
doubleget(a, a_ptr);
doubleget(b, b_ptr);
Import changeset
2000-07-31 21:29:14 +02:00
}
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
#define DBL_EXP_DIG (sizeof(double)*8-DBL_MANT_DIG)
/* The following should work for IEEE */
void Field_double::sort_string(char *to,uint length __attribute__((unused)))
{
double nr;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
float8get(nr,ptr);
}
else
#endif
Use new bitmap interface Patches for Armstrong Removed warnings when using REPAIR TABLE .. EXTENDED
2001-02-07 16:42:20 +01:00
doubleget(nr,ptr);
Import changeset
2000-07-31 21:29:14 +02:00
change_double_for_sort(nr, (byte*) to);
}
void Field_double::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
Import changeset
2000-07-31 21:29:14 +02:00
if (dec == NOT_FIXED_DEC)
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("double"));
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
else
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
{
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*) res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"double(%d,%d)",(int) field_length,dec));
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
add_zerofill_and_unsigned(res);
}
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
/*
TIMESTAMP type.
Holds datetime values in range from 1970-01-01 00:00:01 UTC to
2038-01-01 00:00:00 UTC stored as number of seconds since Unix
Epoch in UTC.
Up to one of timestamps columns in the table can be automatically
set on row update and/or have NOW() as default value.
TABLE::timestamp_field points to Field object for such timestamp with
auto-set-on-update. TABLE::time_stamp holds offset in record + 1 for this
field, and is used by handler code which performs updates required.
Actually SQL-99 says that we should allow niladic functions (like NOW())
as defaults for any field. Current limitations (only NOW() and only
for one TIMESTAMP field) are because of restricted binary .frm format
and should go away in the future.
Also because of this limitation of binary .frm format we use 5 different
unireg_check values with TIMESTAMP field to distinguish various cases of
DEFAULT or ON UPDATE values. These values are:
TIMESTAMP_OLD_FIELD - old timestamp, if there was not any fields with
auto-set-on-update (or now() as default) in this table before, then this
field has NOW() as default and is updated when row changes, else it is
Precision Math implementation
2005-02-08 23:50:45 +01:00
field which has 0 as default value and is not automatically updated.
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
TIMESTAMP_DN_FIELD - field with NOW() as default but not set on update
automatically (TIMESTAMP DEFAULT NOW())
TIMESTAMP_UN_FIELD - field which is set on update automatically but has not
NOW() as default (but it may has 0 or some other const timestamp as
default) (TIMESTAMP ON UPDATE NOW()).
TIMESTAMP_DNUN_FIELD - field which has now() as default and is auto-set on
update. (TIMESTAMP DEFAULT NOW() ON UPDATE NOW())
NONE - field which is not auto-set on update with some other than NOW()
default value (TIMESTAMP DEFAULT 0).
Note that TIMESTAMP_OLD_FIELD's are never created explicitly now, they are
left only for preserving ability to read old tables. Such fields replaced
with their newer analogs in CREATE TABLE and in SHOW CREATE TABLE. This is
because we want to prefer NONE unireg_check before TIMESTAMP_OLD_FIELD for
"TIMESTAMP DEFAULT 'Const'" field. (Old timestamps allowed such
specification too but ignored default value for first timestamp, which of
course is non-standard.) In most cases user won't notice any change, only
exception is different behavior of old/new timestamps during ALTER TABLE.
*/
Import changeset
2000-07-31 21:29:14 +02:00
Field_timestamp::Field_timestamp(char *ptr_arg, uint32 len_arg,
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
uchar *null_ptr_arg, uchar null_bit_arg,
Import changeset
2000-07-31 21:29:14 +02:00
enum utype unireg_check_arg,
const char *field_name_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
TABLE_SHARE *share,
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
CHARSET_INFO *cs)
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
:Field_str(ptr_arg, 19, null_ptr_arg, null_bit_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check_arg, field_name_arg, cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fix for bug #4492. TIMESTAMP columns should be unsigned to preserve compatibility with 4.0 (Or else InnoDB will return different internal TIMESTAMP values when user upgrades to 4.1). Altough this fix will introduce problems with early 4.1 -> 4.1 upgrades (tables with TIMESTAMP field should be reloaded using mysqldump) it will allow easy 4.0 -> 4.1 upgrade (which is more important since 4.1 is still beta).
2004-07-28 08:49:21 +02:00
/* For 4.0 MYD and 4.0 InnoDB compatibility */
flags|= ZEROFILL_FLAG | UNSIGNED_FLAG;
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
if (!share->timestamp_field && unireg_check != NONE)
Import changeset
2000-07-31 21:29:14 +02:00
{
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
/* This timestamp has auto-update */
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
share->timestamp_field= this;
flags|= TIMESTAMP_FLAG;
Import changeset
2000-07-31 21:29:14 +02:00
}
}
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
/*
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
Get auto-set type for TIMESTAMP field.
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
SYNOPSIS
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
get_auto_set_type()
DESCRIPTION
Returns value indicating during which operations this TIMESTAMP field
should be auto-set to current timestamp.
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
*/
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
timestamp_auto_set_type Field_timestamp::get_auto_set_type() const
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
{
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
switch (unireg_check)
{
case TIMESTAMP_DN_FIELD:
return TIMESTAMP_AUTO_SET_ON_INSERT;
case TIMESTAMP_UN_FIELD:
return TIMESTAMP_AUTO_SET_ON_UPDATE;
Fixed small bug in handling of pre-4.1 TIMESTAMP columns which was introduced during implementation of TIMESTAMP columns, which are able to store NULLs (Unfortunately it is impossible to write test case for this. Kudos to Holyfoot for noticing it!)
2004-10-05 19:23:38 +02:00
case TIMESTAMP_OLD_FIELD:
/*
Precision Math implementation
2005-02-08 23:50:45 +01:00
Although we can have several such columns in legacy tables this
Fixed small bug in handling of pre-4.1 TIMESTAMP columns which was introduced during implementation of TIMESTAMP columns, which are able to store NULLs (Unfortunately it is impossible to write test case for this. Kudos to Holyfoot for noticing it!)
2004-10-05 19:23:38 +02:00
function should be called only for first of them (i.e. the one
having auto-set property).
*/
DBUG_ASSERT(table->timestamp_field == this);
/* Fall-through */
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
case TIMESTAMP_DNUN_FIELD:
return TIMESTAMP_AUTO_SET_ON_BOTH;
default:
/*
Normally this function should not be called for TIMESTAMPs without
auto-set property.
*/
DBUG_ASSERT(0);
return TIMESTAMP_NO_AUTO_SET;
}
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_timestamp::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
TIME l_time;
my_time_t tmp= 0;
int error;
bool have_smth_to_conv;
Fixes during review of new pushed code Change bool in C code to my_bool Added to mysqltest --enable_parsning and --disable_parsing to avoid to have to comment parts of tests Added comparison of LEX_STRING's and use this to compare file types for view and trigger files.
2005-07-31 11:49:55 +02:00
my_bool in_dst_time_gap;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
THD *thd= table->in_use;
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
/* We don't want to store invalid or fuzzy datetime values in TIMESTAMP */
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
have_smth_to_conv= (str_to_datetime(from, len, &l_time,
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
(table->in_use->variables.sql_mode &
MODE_NO_ZERO_DATE) |
MODE_NO_ZERO_IN_DATE, &error) >
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
MYSQL_TIMESTAMP_ERROR);
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (error || !have_smth_to_conv)
{
error= 1;
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_DATETIME, 1);
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
}
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
/* Only convert a correct date (not a zero date) */
if (have_smth_to_conv && l_time.month)
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
{
if (!(tmp= TIME_to_timestamp(thd, &l_time, &in_dst_time_gap)))
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
ER_WARN_DATA_OUT_OF_RANGE,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_DATETIME, !error);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
error= 1;
}
else if (in_dst_time_gap)
{
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
ER_WARN_INVALID_TIMESTAMP,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_DATETIME, !error);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
error= 1;
}
}
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,tmp);
}
else
#endif
longstore(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_timestamp::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if (nr < 0 || nr > 99991231235959.0)
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_OUT_OF_RANGE,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
nr, MYSQL_TIMESTAMP_DATETIME);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
nr= 0; // Avoid overflow on buff
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
error|= Field_timestamp::store((longlong) rint(nr), FALSE);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_timestamp::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
TIME l_time;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
my_time_t timestamp= 0;
int error;
Fixes during review of new pushed code Change bool in C code to my_bool Added to mysqltest --enable_parsning and --disable_parsing to avoid to have to comment parts of tests Added comparison of LEX_STRING's and use this to compare file types for view and trigger files.
2005-07-31 11:49:55 +02:00
my_bool in_dst_time_gap;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
THD *thd= table->in_use;
Import changeset
2000-07-31 21:29:14 +02:00
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
/* We don't want to store invalid or fuzzy datetime values in TIMESTAMP */
Fix date conversion on 32-bit machines.
2005-07-20 19:16:43 +02:00
longlong tmp= number_to_datetime(nr, &l_time, (thd->variables.sql_mode &
MODE_NO_ZERO_DATE) |
MODE_NO_ZERO_IN_DATE, &error);
Fixes during review of new pushed code Change bool in C code to my_bool Added to mysqltest --enable_parsning and --disable_parsing to avoid to have to comment parts of tests Added comparison of LEX_STRING's and use this to compare file types for view and trigger files.
2005-07-31 11:49:55 +02:00
if (tmp == LL(-1))
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
{
error= 2;
}
if (!error && tmp)
Import changeset
2000-07-31 21:29:14 +02:00
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if (!(timestamp= TIME_to_timestamp(thd, &l_time, &in_dst_time_gap)))
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_OUT_OF_RANGE,
nr, MYSQL_TIMESTAMP_DATETIME, 1);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
error= 1;
}
if (in_dst_time_gap)
{
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
ER_WARN_INVALID_TIMESTAMP,
nr, MYSQL_TIMESTAMP_DATETIME, 1);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
error= 1;
}
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
} else if (error)
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
WARN_DATA_TRUNCATED,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
nr, MYSQL_TIMESTAMP_DATETIME, 1);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,timestamp);
}
else
#endif
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
longstore(ptr,(uint32) timestamp);
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
Merge with 4.0.17
2003-12-17 16:35:34 +01:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_timestamp::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
return (double) Field_timestamp::val_int();
}
longlong Field_timestamp::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
uint32 temp;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
TIME time_tmp;
THD *thd= table->in_use;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
temp=uint4korr(ptr);
else
#endif
longget(temp,ptr);
if (temp == 0L) // No time
return(0); /* purecov: inspected */
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
thd->variables.time_zone->gmt_sec_to_TIME(&time_tmp, (my_time_t)temp);
thd->time_zone_used= 1;
return time_tmp.year * LL(10000000000) + time_tmp.month * LL(100000000) +
time_tmp.day * 1000000L + time_tmp.hour * 10000L +
time_tmp.minute * 100 + time_tmp.second;
Import changeset
2000-07-31 21:29:14 +02:00
}
post-review fixes HA_EXTRA_WRITE_CACHE now part of start_bulk_insert() test cleanups
2004-04-07 16:04:28 +02:00
String *Field_timestamp::val_str(String *val_buffer, String *val_ptr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
uint32 temp, temp2;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
TIME time_tmp;
THD *thd= table->in_use;
Merge with 4.0.19
2004-05-05 16:05:24 +02:00
char *to;
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->alloc(field_length+1);
Merge with 4.0.19
2004-05-05 16:05:24 +02:00
to= (char*) val_buffer->ptr();
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
val_buffer->length(field_length);
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
temp=uint4korr(ptr);
else
#endif
longget(temp,ptr);
if (temp == 0L)
{ /* Zero time is "000000" */
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
val_ptr->set(STRING_WITH_LEN("0000-00-00 00:00:00"), &my_charset_bin);
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
return val_ptr;
Import changeset
2000-07-31 21:29:14 +02:00
}
Move latin1 into a separarte file
2003-01-29 14:31:20 +01:00
val_buffer->set_charset(&my_charset_bin); // Safety
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
thd->variables.time_zone->gmt_sec_to_TIME(&time_tmp,(my_time_t)temp);
thd->time_zone_used= 1;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
temp= time_tmp.year % 100;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
if (temp < YY_PART_YEAR)
{
*to++= '2';
*to++= '0';
}
else
{
*to++= '1';
*to++= '9';
}
temp2=temp/10; temp=temp-temp2*10;
*to++= (char) ('0'+(char) (temp2));
*to++= (char) ('0'+(char) (temp));
*to++= '-';
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
temp=time_tmp.month;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
temp2=temp/10; temp=temp-temp2*10;
*to++= (char) ('0'+(char) (temp2));
*to++= (char) ('0'+(char) (temp));
*to++= '-';
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
temp=time_tmp.day;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
temp2=temp/10; temp=temp-temp2*10;
*to++= (char) ('0'+(char) (temp2));
*to++= (char) ('0'+(char) (temp));
*to++= ' ';
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
temp=time_tmp.hour;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
temp2=temp/10; temp=temp-temp2*10;
*to++= (char) ('0'+(char) (temp2));
*to++= (char) ('0'+(char) (temp));
*to++= ':';
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
temp=time_tmp.minute;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
temp2=temp/10; temp=temp-temp2*10;
*to++= (char) ('0'+(char) (temp2));
*to++= (char) ('0'+(char) (temp));
*to++= ':';
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
temp=time_tmp.second;
Fixes for binary protocol (complement to last push) Changed timestamp to return string in YYYY-MM-DD HH:MM:SS format. DATE_ADD() and related functions now returns correct DATE/DATETIME type depending on argument types. Now all tests passes, still some work left to remove warnings in log files from mysql-test-run
2002-12-14 16:43:01 +01:00
temp2=temp/10; temp=temp-temp2*10;
*to++= (char) ('0'+(char) (temp2));
*to++= (char) ('0'+(char) (temp));
*to= 0;
Import changeset
2000-07-31 21:29:14 +02:00
return val_buffer;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
bool Field_timestamp::get_date(TIME *ltime, uint fuzzydate)
Import changeset
2000-07-31 21:29:14 +02:00
{
long temp;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
THD *thd= table->in_use;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
temp=uint4korr(ptr);
else
#endif
longget(temp,ptr);
if (temp == 0L)
{ /* Zero time is "000000" */
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (fuzzydate & TIME_NO_ZERO_DATE)
Portability fixes Fix for consistent 0000-00-00 date handling Close + open binary logs on flush tables Fix for AVG() in HAVING.
2001-10-31 17:27:49 +01:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
bzero((char*) ltime,sizeof(*ltime));
}
else
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
thd->variables.time_zone->gmt_sec_to_TIME(ltime, (my_time_t)temp);
thd->time_zone_used= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
return 0;
}
bool Field_timestamp::get_time(TIME *ltime)
{
Portability fixes Fix for consistent 0000-00-00 date handling Close + open binary logs on flush tables Fix for AVG() in HAVING.
2001-10-31 17:27:49 +01:00
return Field_timestamp::get_date(ltime,0);
Import changeset
2000-07-31 21:29:14 +02:00
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_timestamp::send_binary(Protocol *protocol)
{
TIME tm;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
Field_timestamp::get_date(&tm, 0);
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
return protocol->store(&tm);
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_timestamp::cmp(const char *a_ptr, const char *b_ptr)
{
int32 a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
a=sint4korr(a_ptr);
b=sint4korr(b_ptr);
}
else
#endif
{
longget(a,a_ptr);
longget(b,b_ptr);
}
return ((uint32) a < (uint32) b) ? -1 : ((uint32) a > (uint32) b) ? 1 : 0;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
void Field_timestamp::sort_string(char *to,uint length __attribute__((unused)))
{
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (!table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
to[0] = ptr[0];
to[1] = ptr[1];
to[2] = ptr[2];
to[3] = ptr[3];
}
else
#endif
{
to[0] = ptr[3];
to[1] = ptr[2];
to[2] = ptr[1];
to[3] = ptr[0];
}
}
void Field_timestamp::sql_type(String &res) const
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("timestamp"));
Import changeset
2000-07-31 21:29:14 +02:00
}
void Field_timestamp::set_time()
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
long tmp= (long) table->in_use->query_start();
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
set_notnull();
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,tmp);
}
else
#endif
longstore(ptr,tmp);
}
/****************************************************************************
** time type
** In string context: HH:MM:SS
** In number context: HHMMSS
** Stored as a 3 byte unsigned int
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_time::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
TIME ltime;
long tmp;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
int error;
if (str_to_time(from, len, &ltime, &error))
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
{
Import changeset
2000-07-31 21:29:14 +02:00
tmp=0L;
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
error= 2;
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_TIME, 1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
else
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if (error)
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
WARN_DATA_TRUNCATED,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_TIME, 1);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
if (ltime.month)
ltime.day=0;
tmp=(ltime.day*24L+ltime.hour)*10000L+(ltime.minute*100+ltime.second);
if (tmp > 8385959)
{
tmp=8385959;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_OUT_OF_RANGE,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_TIME, !error);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
if (error > 1)
error= 2;
Import changeset
2000-07-31 21:29:14 +02:00
}
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
if (ltime.neg)
tmp= -tmp;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
error |= Field_time::store((longlong) tmp, FALSE);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
int Field_time::store_time(TIME *ltime, timestamp_type type)
{
long tmp= ((ltime->month ? 0 : ltime->day * 24L) + ltime->hour) * 10000L +
(ltime->minute * 100 + ltime->second);
if (ltime->neg)
tmp= -tmp;
return Field_time::store((longlong) tmp);
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_time::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
long tmp;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if (nr > 8385959.0)
{
tmp=8385959L;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ER_WARN_DATA_OUT_OF_RANGE, nr, MYSQL_TIMESTAMP_TIME);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else if (nr < -8385959.0)
{
tmp= -8385959L;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ER_WARN_DATA_OUT_OF_RANGE, nr, MYSQL_TIMESTAMP_TIME);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
tmp=(long) floor(fabs(nr)); // Remove fractions
if (nr < 0)
tmp= -tmp;
if (tmp % 100 > 59 || tmp/100 % 100 > 59)
{
tmp=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ER_WARN_DATA_OUT_OF_RANGE, nr,
MYSQL_TIMESTAMP_TIME);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
}
int3store(ptr,tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_time::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
long tmp;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if (nr < (longlong) -8385959L && !unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
tmp= -8385959L;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ER_WARN_DATA_OUT_OF_RANGE, nr,
MYSQL_TIMESTAMP_TIME, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
else if (nr > (longlong) 8385959 || nr < 0 && unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
tmp=8385959L;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ER_WARN_DATA_OUT_OF_RANGE, nr,
MYSQL_TIMESTAMP_TIME, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
tmp=(long) nr;
if (tmp % 100 > 59 || tmp/100 % 100 > 59)
{
tmp=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ER_WARN_DATA_OUT_OF_RANGE, nr,
MYSQL_TIMESTAMP_TIME, 1);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
}
int3store(ptr,tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_time::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 j= (uint32) uint3korr(ptr);
Import changeset
2000-07-31 21:29:14 +02:00
return (double) j;
}
longlong Field_time::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
return (longlong) sint3korr(ptr);
}
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
/*
This function is multi-byte safe as the result string is always of type
my_charset_bin
*/
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_time::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Scrum task 835 - text-to-datetime conversion function
2003-10-20 10:24:18 +02:00
TIME ltime;
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
val_buffer->alloc(19);
Import changeset
2000-07-31 21:29:14 +02:00
long tmp=(long) sint3korr(ptr);
Scrum task 835 - text-to-datetime conversion function
2003-10-20 10:24:18 +02:00
ltime.neg= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if (tmp < 0)
{
tmp= -tmp;
Scrum task 835 - text-to-datetime conversion function
2003-10-20 10:24:18 +02:00
ltime.neg= 1;
}
ltime.day= (uint) 0;
ltime.hour= (uint) (tmp/10000);
ltime.minute= (uint) (tmp/100 % 100);
ltime.second= (uint) (tmp % 100);
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
make_time((DATE_TIME_FORMAT*) 0, &ltime, val_buffer);
Import changeset
2000-07-31 21:29:14 +02:00
return val_buffer;
}
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
After merge fixes Return NULL if a time argument is given to date_add(). (Warning will be shown after Dimitri's timezone patch is pushed)
2004-06-21 09:21:20 +02:00
/*
Precision Math implementation
2005-02-08 23:50:45 +01:00
Normally we would not consider 'time' as a valid date, but we allow
After merge fixes Return NULL if a time argument is given to date_add(). (Warning will be shown after Dimitri's timezone patch is pushed)
2004-06-21 09:21:20 +02:00
get_date() here to be able to do things like
DATE_FORMAT(time, "%l.%i %p")
*/
bool Field_time::get_date(TIME *ltime, uint fuzzydate)
Fix for bug #4036 multiple SELECT DATE_FORMAT, incorrect results
2004-06-07 18:35:05 +02:00
{
After merge fixes Return NULL if a time argument is given to date_add(). (Warning will be shown after Dimitri's timezone patch is pushed)
2004-06-21 09:21:20 +02:00
long tmp;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (!(fuzzydate & TIME_FUZZY_DATE))
After merge fixes Return NULL if a time argument is given to date_add(). (Warning will be shown after Dimitri's timezone patch is pushed)
2004-06-21 09:21:20 +02:00
{
Field_geom should have same max_length as Field_blob Updated tests After merge fix
2004-06-21 10:27:40 +02:00
push_warning_printf(table->in_use, MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_OUT_OF_RANGE,
ER(ER_WARN_DATA_OUT_OF_RANGE), field_name,
table->in_use->row_count);
After merge fixes Return NULL if a time argument is given to date_add(). (Warning will be shown after Dimitri's timezone patch is pushed)
2004-06-21 09:21:20 +02:00
return 1;
}
tmp=(long) sint3korr(ptr);
Fix for bug #4036 multiple SELECT DATE_FORMAT, incorrect results
2004-06-07 18:35:05 +02:00
ltime->neg=0;
if (tmp < 0)
{
ltime->neg= 1;
tmp=-tmp;
}
ltime->hour=tmp/10000;
tmp-=ltime->hour*10000;
ltime->minute= tmp/100;
ltime->second= tmp % 100;
ltime->year= ltime->month= ltime->day= ltime->second_part= 0;
return 0;
}
Merge with 4.0.21
2004-06-18 03:38:58 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
bool Field_time::get_time(TIME *ltime)
{
long tmp=(long) sint3korr(ptr);
ltime->neg=0;
if (tmp < 0)
{
ltime->neg= 1;
tmp=-tmp;
}
Internal commit
2003-06-23 09:56:44 +02:00
ltime->day= 0;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
ltime->hour= (int) (tmp/10000);
Fixes for FULLTEXT and TIME type
2000-09-13 00:08:34 +02:00
tmp-=ltime->hour*10000;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
ltime->minute= (int) tmp/100;
ltime->second= (int) tmp % 100;
Import changeset
2000-07-31 21:29:14 +02:00
ltime->second_part=0;
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ltime->time_type= MYSQL_TIMESTAMP_TIME;
Import changeset
2000-07-31 21:29:14 +02:00
return 0;
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
bool Field_time::send_binary(Protocol *protocol)
{
TIME tm;
Field_time::get_time(&tm);
A lot of fixes for prepared statements (PS): New mysqltest that can run mysqltest with PS Added support for ZEROFILL in PS Fixed crash when one called mysql_stmt_store_result() without a preceding mysql_stmt_bind_result() Updated test cases to support --ps-protocol (Some tests are still run using old protocol) Fixed crash in PS when using SELECT * FROM t1 NATURAL JOIN t2... Fixed crash in PS when using sub queries Create table didn't signal when table was created. This could cause a "DROP TABLE created_table" in another thread to wait "forever" Fixed wrong permissions check in PS and multi-table updates (one could get permission denied for legal quries) Fix for PS and SELECT ... PROCEDURE Reset all warnings when executing a new PS query group_concat(...ORDER BY) didn't work with PS Fixed problem with test suite when not using innodb
2004-10-26 18:30:01 +02:00
tm.day= tm.hour/24; // Move hours to days
tm.hour-= tm.day*24;
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
return protocol->store_time(&tm);
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_time::cmp(const char *a_ptr, const char *b_ptr)
{
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
int32 a,b;
a=(int32) sint3korr(a_ptr);
b=(int32) sint3korr(b_ptr);
Import changeset
2000-07-31 21:29:14 +02:00
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
void Field_time::sort_string(char *to,uint length __attribute__((unused)))
{
to[0] = (uchar) (ptr[2] ^ 128);
to[1] = ptr[1];
to[2] = ptr[0];
}
void Field_time::sql_type(String &res) const
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("time"));
Import changeset
2000-07-31 21:29:14 +02:00
}
/****************************************************************************
** year type
** Save in a byte the year 0, 1901->2155
** Can handle 2 byte or 4 byte years!
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_year::store(const char *from, uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
char *end;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
int error;
long nr= my_strntol(cs, from, len, 10, &end, &error);
Import changeset
2000-07-31 21:29:14 +02:00
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (nr < 0 || nr >= 100 && nr <= 1900 || nr > 2155 || error)
Import changeset
2000-07-31 21:29:14 +02:00
{
*ptr=0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (table->in_use->count_cuted_fields && check_int(from,len,end,cs))
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
if (nr != 0 || len != 4)
{
if (nr < YY_PART_YEAR)
nr+=100; // 2000 - 2069
else if (nr > 1900)
nr-= 1900;
}
*ptr= (char) (unsigned char) nr;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.17
2003-12-17 16:35:34 +01:00
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_year::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
if (nr < 0.0 || nr >= 2155.0)
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
(void) Field_year::store((longlong) -1, FALSE);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
return Field_year::store((longlong) nr, FALSE);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_year::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
if (nr < 0 || nr >= 100 && nr <= 1900 || nr > 2155)
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
*ptr= 0;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
if (nr != 0 || field_length != 4) // 0000 -> 0; 00 -> 2000
{
if (nr < YY_PART_YEAR)
nr+=100; // 2000 - 2069
else if (nr > 1900)
nr-= 1900;
}
*ptr= (char) (unsigned char) nr;
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
bool Field_year::send_binary(Protocol *protocol)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
ulonglong tmp= Field_year::val_int();
Fix date,time,timestamp,year - binary protocol
2003-01-04 08:37:16 +01:00
return protocol->store_short(tmp);
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
double Field_year::val_real(void)
{
return (double) Field_year::val_int();
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
longlong Field_year::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
int tmp= (int) ((uchar*) ptr)[0];
if (field_length != 4)
tmp%=100; // Return last 2 char
else if (tmp)
tmp+=1900;
return (longlong) tmp;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_year::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
val_buffer->alloc(5);
val_buffer->length(field_length);
char *to=(char*) val_buffer->ptr();
sprintf(to,field_length == 2 ? "%02d" : "%04d",(int) Field_year::val_int());
return val_buffer;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
void Field_year::sql_type(String &res) const
{
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
res.length(cs->cset->snprintf(cs,(char*)res.ptr(),res.alloced_length(),
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
"year(%d)",(int) field_length));
Import changeset
2000-07-31 21:29:14 +02:00
}
/****************************************************************************
** date type
** In string context: YYYY-MM-DD
** In number context: YYYYMMDD
** Stored as a 4 byte unsigned int
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_date::store(const char *from, uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
TIME l_time;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 tmp;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
int error;
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (str_to_datetime(from, len, &l_time, TIME_FUZZY_DATE |
(table->in_use->variables.sql_mode &
(MODE_NO_ZERO_IN_DATE | MODE_NO_ZERO_DATE |
MODE_INVALID_DATES)),
&error) <= MYSQL_TIMESTAMP_ERROR)
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
tmp= 0;
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
error= 2;
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
else
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
tmp=(uint32) l_time.year*10000L + (uint32) (l_time.month*100+l_time.day);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if (error)
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_DATE, 1);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,tmp);
}
else
#endif
longstore(ptr,tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_date::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fix warnings/compilation failures reported by Jan Lindstrom
2005-07-20 12:17:06 +02:00
longlong tmp;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if (nr >= 19000000000000.0 && nr <= 99991231235959.0)
nr=floor(nr/1000000.0); // Timestamp to date
if (nr < 0.0 || nr > 99991231.0)
{
Fix warnings/compilation failures reported by Jan Lindstrom
2005-07-20 12:17:06 +02:00
tmp= LL(0);
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_OUT_OF_RANGE,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
nr, MYSQL_TIMESTAMP_DATE);
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
Fix warnings/compilation failures reported by Jan Lindstrom
2005-07-20 12:17:06 +02:00
tmp= (longlong) rint(nr);
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Fixes bug #15634. Eliminates compiler warning 'all return paths are recursive in Field_date::store function'. Though the Field_date::store function almost unused when protocol_version=10 additional check was added into it to store 4byte dates properly. Effective test routine is not available so far due to protocol_version is not a dynamic property and can not be modified with mysql-test script.
2005-12-28 17:40:37 +01:00
return Field_date::store(tmp, TRUE);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_date::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
TIME not_used;
int error;
longlong initial_nr= nr;
nr= number_to_datetime(nr, &not_used, (TIME_FUZZY_DATE |
(table->in_use->variables.sql_mode &
(MODE_NO_ZERO_IN_DATE |
MODE_NO_ZERO_DATE |
MODE_INVALID_DATES))), &error);
Fixes during review of new pushed code Change bool in C code to my_bool Added to mysqltest --enable_parsning and --disable_parsing to avoid to have to comment parts of tests Added comparison of LEX_STRING's and use this to compare file types for view and trigger files.
2005-07-31 11:49:55 +02:00
if (nr == LL(-1))
Import changeset
2000-07-31 21:29:14 +02:00
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
nr= 0;
error= 2;
Import changeset
2000-07-31 21:29:14 +02:00
}
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
Fixes bug #15634. Eliminates compiler warning 'all return paths are recursive in Field_date::store function'. Though the Field_date::store function almost unused when protocol_version=10 additional check was added into it to store 4byte dates properly. Effective test routine is not available so far due to protocol_version is not a dynamic property and can not be modified with mysql-test script.
2005-12-28 17:40:37 +01:00
if (nr >= 19000000000000.0 && nr <= 99991231235959.0)
Fixed compiler warnings from gcc 4.0.2: - Added empty constructors and virtual destructors to many classes and structs - Removed some usage of the offsetof() macro to instead use C++ class pointers
2006-02-25 16:46:30 +01:00
nr= (longlong) floor(nr/1000000.0); // Timestamp to date
Fixes bug #15634. Eliminates compiler warning 'all return paths are recursive in Field_date::store function'. Though the Field_date::store function almost unused when protocol_version=10 additional check was added into it to store 4byte dates properly. Effective test routine is not available so far due to protocol_version is not a dynamic property and can not be modified with mysql-test script.
2005-12-28 17:40:37 +01:00
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
if (error)
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
error == 2 ? ER_WARN_DATA_OUT_OF_RANGE :
WARN_DATA_TRUNCATED, initial_nr,
MYSQL_TIMESTAMP_DATETIME, 1);
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
int4store(ptr, nr);
Import changeset
2000-07-31 21:29:14 +02:00
}
else
#endif
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
longstore(ptr, nr);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
bool Field_date::send_binary(Protocol *protocol)
{
longlong tmp= Field_date::val_int();
TIME tm;
tm.year= (uint32) tmp/10000L % 10000;
tm.month= (uint32) tmp/100 % 100;
tm.day= (uint32) tmp % 100;
return protocol->store_date(&tm);
}
Import changeset
2000-07-31 21:29:14 +02:00
double Field_date::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
int32 j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint4korr(ptr);
else
#endif
longget(j,ptr);
return (double) (uint32) j;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
longlong Field_date::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
int32 j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint4korr(ptr);
else
#endif
longget(j,ptr);
return (longlong) (uint32) j;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_date::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Scrum task 835 - text-to-datetime conversion function
2003-10-20 10:24:18 +02:00
TIME ltime;
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->alloc(field_length);
int32 tmp;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
tmp=sint4korr(ptr);
else
#endif
longget(tmp,ptr);
Scrum task 835 - text-to-datetime conversion function
2003-10-20 10:24:18 +02:00
ltime.neg= 0;
ltime.year= (int) ((uint32) tmp/10000L % 10000);
ltime.month= (int) ((uint32) tmp/100 % 100);
ltime.day= (int) ((uint32) tmp % 100);
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
make_date((DATE_TIME_FORMAT *) 0, &ltime, val_buffer);
Import changeset
2000-07-31 21:29:14 +02:00
return val_buffer;
}
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
int Field_date::cmp(const char *a_ptr, const char *b_ptr)
{
int32 a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
a=sint4korr(a_ptr);
b=sint4korr(b_ptr);
}
else
#endif
{
longget(a,a_ptr);
longget(b,b_ptr);
}
return ((uint32) a < (uint32) b) ? -1 : ((uint32) a > (uint32) b) ? 1 : 0;
}
void Field_date::sort_string(char *to,uint length __attribute__((unused)))
{
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (!table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
to[0] = ptr[0];
to[1] = ptr[1];
to[2] = ptr[2];
to[3] = ptr[3];
}
else
#endif
{
to[0] = ptr[3];
to[1] = ptr[2];
to[2] = ptr[1];
to[3] = ptr[0];
}
}
void Field_date::sql_type(String &res) const
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("date"));
Import changeset
2000-07-31 21:29:14 +02:00
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
** The new date type
** This is identical to the old date type, but stored on 3 bytes instead of 4
** In number context: YYYYMMDD
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_newdate::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
TIME l_time;
long tmp;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
int error;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (str_to_datetime(from, len, &l_time,
(TIME_FUZZY_DATE |
(table->in_use->variables.sql_mode &
(MODE_NO_ZERO_IN_DATE | MODE_NO_ZERO_DATE |
MODE_INVALID_DATES))),
&error) <= MYSQL_TIMESTAMP_ERROR)
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
tmp= 0L;
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
error= 2;
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
else
tmp= l_time.day + l_time.month*32 + l_time.year*16*32;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if (error)
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_DATE, 1);
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
int3store(ptr,tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_newdate::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
if (nr < 0.0 || nr > 99991231235959.0)
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
{
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
int3store(ptr,(int32) 0);
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
WARN_DATA_TRUNCATED, nr, MYSQL_TIMESTAMP_DATE);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 1;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
return Field_newdate::store((longlong) rint(nr), FALSE);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_newdate::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
TIME l_time;
Fix date conversion on 32-bit machines.
2005-07-20 19:16:43 +02:00
longlong tmp;
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
int error;
Fixes during review of new pushed code Change bool in C code to my_bool Added to mysqltest --enable_parsning and --disable_parsing to avoid to have to comment parts of tests Added comparison of LEX_STRING's and use this to compare file types for view and trigger files.
2005-07-31 11:49:55 +02:00
if (number_to_datetime(nr, &l_time,
(TIME_FUZZY_DATE |
(table->in_use->variables.sql_mode &
(MODE_NO_ZERO_IN_DATE | MODE_NO_ZERO_DATE |
MODE_INVALID_DATES))),
&error) == LL(-1))
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
{
tmp= 0L;
error= 2;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
tmp= l_time.day + l_time.month*32 + l_time.year*16*32;
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
if (error)
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
error == 2 ? ER_WARN_DATA_OUT_OF_RANGE :
WARN_DATA_TRUNCATED,nr,MYSQL_TIMESTAMP_DATE, 1);
int3store(ptr,tmp);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
int Field_newdate::store_time(TIME *ltime,timestamp_type type)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
long tmp;
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
int error= 0;
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
if (type == MYSQL_TIMESTAMP_DATE || type == MYSQL_TIMESTAMP_DATETIME)
Import changeset
2000-07-31 21:29:14 +02:00
tmp=ltime->year*16*32+ltime->month*32+ltime->day;
else
{
tmp=0;
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
error= 1;
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Import changeset
2000-07-31 21:29:14 +02:00
}
int3store(ptr,tmp);
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
bool Field_newdate::send_binary(Protocol *protocol)
{
TIME tm;
Field_newdate::get_date(&tm,0);
return protocol->store_date(&tm);
}
Import changeset
2000-07-31 21:29:14 +02:00
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
double Field_newdate::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
return (double) Field_newdate::val_int();
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
longlong Field_newdate::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
ulong j= uint3korr(ptr);
Import changeset
2000-07-31 21:29:14 +02:00
j= (j % 32L)+(j / 32L % 16L)*100L + (j/(16L*32L))*10000L;
return (longlong) j;
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
String *Field_newdate::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->alloc(field_length);
val_buffer->length(field_length);
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 tmp=(uint32) uint3korr(ptr);
Import changeset
2000-07-31 21:29:14 +02:00
int part;
char *pos=(char*) val_buffer->ptr()+10;
/* Open coded to get more speed */
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*pos--=0; // End NULL
Import changeset
2000-07-31 21:29:14 +02:00
part=(int) (tmp & 31);
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*pos--= (char) ('0'+part%10);
*pos--= (char) ('0'+part/10);
*pos--= '-';
Import changeset
2000-07-31 21:29:14 +02:00
part=(int) (tmp >> 5 & 15);
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*pos--= (char) ('0'+part%10);
*pos--= (char) ('0'+part/10);
*pos--= '-';
Import changeset
2000-07-31 21:29:14 +02:00
part=(int) (tmp >> 9);
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*pos--= (char) ('0'+part%10); part/=10;
*pos--= (char) ('0'+part%10); part/=10;
*pos--= (char) ('0'+part%10); part/=10;
*pos= (char) ('0'+part);
Import changeset
2000-07-31 21:29:14 +02:00
return val_buffer;
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
bool Field_newdate::get_date(TIME *ltime,uint fuzzydate)
Import changeset
2000-07-31 21:29:14 +02:00
{
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 tmp=(uint32) uint3korr(ptr);
Import changeset
2000-07-31 21:29:14 +02:00
ltime->day= tmp & 31;
ltime->month= (tmp >> 5) & 15;
ltime->year= (tmp >> 9);
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ltime->time_type= MYSQL_TIMESTAMP_DATE;
Fix for BUG#1960 "date_format() returns spurious '-' for valid dates". It was a forgotten ltime->neg=0 (neg was the only forgotten variable). I scanned field.cc for other places where we would forget to set neg, found none. A test for the bug.
2003-11-26 17:06:24 +01:00
ltime->hour= ltime->minute= ltime->second= ltime->second_part= ltime->neg= 0;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
return ((!(fuzzydate & TIME_FUZZY_DATE) && (!ltime->month || !ltime->day)) ?
1 : 0);
Import changeset
2000-07-31 21:29:14 +02:00
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
bool Field_newdate::get_time(TIME *ltime)
{
Portability fixes Fix for consistent 0000-00-00 date handling Close + open binary logs on flush tables Fix for AVG() in HAVING.
2001-10-31 17:27:49 +01:00
return Field_newdate::get_date(ltime,0);
Import changeset
2000-07-31 21:29:14 +02:00
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
int Field_newdate::cmp(const char *a_ptr, const char *b_ptr)
{
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 a,b;
a=(uint32) uint3korr(a_ptr);
b=(uint32) uint3korr(b_ptr);
Import changeset
2000-07-31 21:29:14 +02:00
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
void Field_newdate::sort_string(char *to,uint length __attribute__((unused)))
{
to[0] = ptr[2];
to[1] = ptr[1];
to[2] = ptr[0];
}
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
void Field_newdate::sql_type(String &res) const
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("date"));
Import changeset
2000-07-31 21:29:14 +02:00
}
/****************************************************************************
** datetime type
** In string context: YYYY-MM-DD HH:MM:DD
** In number context: YYYYMMDDHHMMDD
** Stored as a 8 byte unsigned int. Should sometimes be change to a 6 byte int.
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_datetime::store(const char *from,uint len,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
TIME time_tmp;
int error;
ulonglong tmp= 0;
Enabled more tests of STR_TO_DATE() Fixed that datetime gives warning for zero dates in traditional mode
2005-03-30 22:41:42 +02:00
enum enum_mysql_timestamp_type func_res;
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
Enabled more tests of STR_TO_DATE() Fixed that datetime gives warning for zero dates in traditional mode
2005-03-30 22:41:42 +02:00
func_res= str_to_datetime(from, len, &time_tmp,
(TIME_FUZZY_DATE |
(table->in_use->variables.sql_mode &
(MODE_NO_ZERO_IN_DATE | MODE_NO_ZERO_DATE |
MODE_INVALID_DATES))),
&error);
if ((int) func_res > (int) MYSQL_TIMESTAMP_ERROR)
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
tmp= TIME_to_ulonglong_datetime(&time_tmp);
Enabled more tests of STR_TO_DATE() Fixed that datetime gives warning for zero dates in traditional mode
2005-03-30 22:41:42 +02:00
else
error= 1; // Fix if invalid zero date
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if (error)
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
ER_WARN_DATA_OUT_OF_RANGE,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
from, len, MYSQL_TIMESTAMP_DATETIME, 1);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int8store(ptr,tmp);
}
else
#endif
longlongstore(ptr,tmp);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_datetime::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if (nr < 0.0 || nr > 99991231235959.0)
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
ER_WARN_DATA_OUT_OF_RANGE,
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
nr, MYSQL_TIMESTAMP_DATETIME);
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
nr= 0.0;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
error= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
error|= Field_datetime::store((longlong) rint(nr), FALSE);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_datetime::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Fix undeterministic behaviour of year check if we failed to classify integer as datetime in Field_datetime::store(). Stylistic clean-ups.
2003-12-04 17:02:48 +01:00
TIME not_used;
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
int error;
longlong initial_nr= nr;
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
nr= number_to_datetime(nr, &not_used, (TIME_FUZZY_DATE |
(table->in_use->variables.sql_mode &
(MODE_NO_ZERO_IN_DATE |
MODE_NO_ZERO_DATE |
MODE_INVALID_DATES))), &error);
Fixes during review of new pushed code Change bool in C code to my_bool Added to mysqltest --enable_parsning and --disable_parsing to avoid to have to comment parts of tests Added comparison of LEX_STRING's and use this to compare file types for view and trigger files.
2005-07-31 11:49:55 +02:00
if (nr == LL(-1))
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
{
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
nr= 0;
error= 2;
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
}
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
Fix number to date conversion so it always honors the NO_ZERO_DATE, NO_ZERO_IN_DATE, and INVALID_DATES bits of SQL_MODE. (Bug #5906)
2005-07-19 01:12:44 +02:00
if (error)
set_datetime_warning(MYSQL_ERROR::WARN_LEVEL_WARN,
error == 2 ? ER_WARN_DATA_OUT_OF_RANGE :
WARN_DATA_TRUNCATED, initial_nr,
MYSQL_TIMESTAMP_DATETIME, 1);
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int8store(ptr,nr);
}
else
#endif
longlongstore(ptr,nr);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.17
2003-12-17 16:35:34 +01:00
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
int Field_datetime::store_time(TIME *ltime,timestamp_type type)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Import changeset
2000-07-31 21:29:14 +02:00
longlong tmp;
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
int error= 0;
Fix for bug #6266 "Invalid DATETIME value is not handled properly". In server we assume that datetime values stored in MYSQL_TIME struct are normalized (and year is not greater than 9999), so we should perform range checks in all places then we convert something to MYSQL_TIME.
2004-11-15 13:44:29 +01:00
/*
We don't perform range checking here since values stored in TIME
structure always fit into DATETIME range.
*/
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
if (type == MYSQL_TIMESTAMP_DATE || type == MYSQL_TIMESTAMP_DATETIME)
Import changeset
2000-07-31 21:29:14 +02:00
tmp=((ltime->year*10000L+ltime->month*100+ltime->day)*LL(1000000)+
(ltime->hour*10000L+ltime->minute*100+ltime->second));
else
{
tmp=0;
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
error= 1;
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Import changeset
2000-07-31 21:29:14 +02:00
}
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int8store(ptr,tmp);
}
else
#endif
longlongstore(ptr,tmp);
Data truncation reporting implementation (libmysql) + post review fixes. Still to do: - deploy my_strtoll10 in limbysql.c - add mysql_options option to switch MYSQL_DATA_TRUNCATED on and off.
2004-12-16 01:15:06 +01:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
bool Field_datetime::send_binary(Protocol *protocol)
{
TIME tm;
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
Field_datetime::get_date(&tm, TIME_FUZZY_DATE);
Binary protocol changes (to monty)
2003-01-03 12:52:53 +01:00
return protocol->store(&tm);
}
Import changeset
2000-07-31 21:29:14 +02:00
double Field_datetime::val_real(void)
{
return (double) Field_datetime::val_int();
}
longlong Field_datetime::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
longlong j;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
j=sint8korr(ptr);
else
#endif
longlongget(j,ptr);
return j;
}
String *Field_datetime::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->alloc(field_length);
val_buffer->length(field_length);
ulonglong tmp;
long part1,part2;
char *pos;
int part3;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
tmp=sint8korr(ptr);
else
#endif
longlongget(tmp,ptr);
/*
Precision Math implementation
2005-02-08 23:50:45 +01:00
Avoid problem with slow longlong arithmetic and sprintf
Import changeset
2000-07-31 21:29:14 +02:00
*/
part1=(long) (tmp/LL(1000000));
part2=(long) (tmp - (ulonglong) part1*LL(1000000));
pos=(char*) val_buffer->ptr()+19;
*pos--=0;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*pos--= (char) ('0'+(char) (part2%10)); part2/=10;
*pos--= (char) ('0'+(char) (part2%10)); part3= (int) (part2 / 10);
*pos--= ':';
*pos--= (char) ('0'+(char) (part3%10)); part3/=10;
*pos--= (char) ('0'+(char) (part3%10)); part3/=10;
*pos--= ':';
*pos--= (char) ('0'+(char) (part3%10)); part3/=10;
*pos--= (char) ('0'+(char) part3);
*pos--= ' ';
*pos--= (char) ('0'+(char) (part1%10)); part1/=10;
*pos--= (char) ('0'+(char) (part1%10)); part1/=10;
*pos--= '-';
*pos--= (char) ('0'+(char) (part1%10)); part1/=10;
*pos--= (char) ('0'+(char) (part1%10)); part3= (int) (part1/10);
*pos--= '-';
*pos--= (char) ('0'+(char) (part3%10)); part3/=10;
*pos--= (char) ('0'+(char) (part3%10)); part3/=10;
*pos--= (char) ('0'+(char) (part3%10)); part3/=10;
*pos=(char) ('0'+(char) part3);
Import changeset
2000-07-31 21:29:14 +02:00
return val_buffer;
}
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
bool Field_datetime::get_date(TIME *ltime, uint fuzzydate)
Import changeset
2000-07-31 21:29:14 +02:00
{
longlong tmp=Field_datetime::val_int();
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 part1,part2;
part1=(uint32) (tmp/LL(1000000));
part2=(uint32) (tmp - (ulonglong) part1*LL(1000000));
Import changeset
2000-07-31 21:29:14 +02:00
Fix for Bug#4030 "Client side conversion string -> date type doesn't work (prepared statements)" and after-review fixes: - str_to_TIME renamed to str_to_datetime to pair with str_to_time - functions str_to_time and str_to_TIME moved to sql-common - send_data_str now supports MYSQL_TYPE_TIME, MYSQL_TIME_DATE, MYSQL_TIME_DATETIME types of user input buffers. - few more comments in the client library - a test case added.
2004-06-24 17:08:36 +02:00
ltime->time_type= MYSQL_TIMESTAMP_DATETIME;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
ltime->neg= 0;
ltime->second_part= 0;
ltime->second= (int) (part2%100);
ltime->minute= (int) (part2/100%100);
ltime->hour= (int) (part2/10000);
ltime->day= (int) (part1%100);
ltime->month= (int) (part1/100%100);
ltime->year= (int) (part1/10000);
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
return (!(fuzzydate & TIME_FUZZY_DATE) && (!ltime->month || !ltime->day)) ? 1 : 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
bool Field_datetime::get_time(TIME *ltime)
{
Portability fixes Fix for consistent 0000-00-00 date handling Close + open binary logs on flush tables Fix for AVG() in HAVING.
2001-10-31 17:27:49 +01:00
return Field_datetime::get_date(ltime,0);
Import changeset
2000-07-31 21:29:14 +02:00
}
int Field_datetime::cmp(const char *a_ptr, const char *b_ptr)
{
longlong a,b;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
a=sint8korr(a_ptr);
b=sint8korr(b_ptr);
}
else
#endif
{
longlongget(a,a_ptr);
longlongget(b,b_ptr);
}
return ((ulonglong) a < (ulonglong) b) ? -1 :
((ulonglong) a > (ulonglong) b) ? 1 : 0;
}
void Field_datetime::sort_string(char *to,uint length __attribute__((unused)))
{
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (!table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
to[0] = ptr[0];
to[1] = ptr[1];
to[2] = ptr[2];
to[3] = ptr[3];
to[4] = ptr[4];
to[5] = ptr[5];
to[6] = ptr[6];
to[7] = ptr[7];
}
else
#endif
{
to[0] = ptr[7];
to[1] = ptr[6];
to[2] = ptr[5];
to[3] = ptr[4];
to[4] = ptr[3];
to[5] = ptr[2];
to[6] = ptr[1];
to[7] = ptr[0];
}
}
void Field_datetime::sql_type(String &res) const
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set_ascii(STRING_WITH_LEN("datetime"));
Import changeset
2000-07-31 21:29:14 +02:00
}
/****************************************************************************
** string type
** A string may be varchar or binary
****************************************************************************/
/* Copy a string and fill with space */
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_string::store(const char *from,uint length,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Minor clean-ups for the previous commit.
2005-04-06 12:12:44 +02:00
int error= 0, well_formed_error;
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
uint32 not_used;
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
String tmpstr(buff,sizeof(buff), &my_charset_bin);
Problem fix: http://bugs.mysql.com/bug.php?id=2366 Wrong utf8 behaviour when data is trancated
2004-02-06 13:59:25 +01:00
uint copy_length;
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
/* See the comment for Field_long::store(long long) */
DBUG_ASSERT(table->in_use == current_thd);
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
/* Convert character set if necessary */
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
if (String::needs_conversion(length, cs, field_charset, &not_used))
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
{
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
uint conv_errors;
tmpstr.copy(from, length, cs, field_charset, &conv_errors);
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
from= tmpstr.ptr();
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
length= tmpstr.length();
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
if (conv_errors)
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
error= 2;
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
}
Problem fix: http://bugs.mysql.com/bug.php?id=2366 Wrong utf8 behaviour when data is trancated
2004-02-06 13:59:25 +01:00
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
/* Make sure we don't break a multibyte sequence or copy malformed data. */
Changed wellformedlen to well_formed_len Fixed that blobs >16M can be inserted/updated Fixed bug when doing CREATE TEMPORARY TABLE ... LIKE
2004-02-17 00:35:17 +01:00
copy_length= field_charset->cset->well_formed_len(field_charset,
Adding a new parameter for well_formed_length to return error. We'll use it for better warnign reporting.
2005-04-06 08:53:15 +02:00
from,from+length,
field_length/
field_charset->mbmaxlen,
Minor clean-ups for the previous commit.
2005-04-06 12:12:44 +02:00
&well_formed_error);
Fix for BUG#15588: String overrun during sp-vars.test The bug appears after implementation of WL#2984 (Make stored routine variables work according to the standard).
2006-01-20 13:59:22 +01:00
memmove(ptr, from, copy_length);
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
/* Append spaces if the string was shorter than the field. */
if (copy_length < field_length)
Problem fix: http://bugs.mysql.com/bug.php?id=2366 Wrong utf8 behaviour when data is trancated
2004-02-06 13:59:25 +01:00
field_charset->cset->fill(field_charset,ptr+copy_length,
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
field_length-copy_length,
type_binary.result, type_binary.test: new file mysql_fix_privilege_tables.sql, mysql_create_system_tables.sh: Adding true BINARY/VARBINARY: fixing "password" type, not to be 0x00-padding. Many files: Adding true BINARY/VARBINARY: fixing tests not to output 0x00 bytes. Adding true BINARY/VARBINARY: new pad_char structure member. ctype-bin.c: Adding true BINARY/VARBINARY: new pad_char structure member. New strnxfrm, with two trailing length bytes. field.cc: Adding true BINARY/VARBINARY.
2005-10-13 16:16:19 +02:00
field_charset->pad_char);
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
/*
Check if we lost any important data (anything in a binary string,
or any non-space in others).
*/
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
if ((copy_length < length) && table->in_use->count_cuted_fields)
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
{
if (binary())
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
error= 2;
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
else
{
const char *end=from+length;
from+= copy_length;
from+= field_charset->cset->scan(field_charset, from, end,
MY_SEQ_SPACES);
if (from != end)
error= 2;
}
Import changeset
2000-07-31 21:29:14 +02:00
}
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
if (error)
A fix (bug #6999: Traditional: Wrong SQLSTATE returned for string truncation).
2005-01-19 09:04:35 +01:00
{
if (table->in_use->abort_on_warning)
set_warning(MYSQL_ERROR::WARN_LEVEL_ERROR, ER_DATA_TOO_LONG, 1);
else
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
A fix (bug #6999: Traditional: Wrong SQLSTATE returned for string truncation).
2005-01-19 09:04:35 +01:00
}
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
/*
Store double value in Field_string or Field_varstring.
SYNOPSIS
after merge fixes
2004-08-19 03:02:09 +02:00
store(double nr)
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
nr number
DESCRIPTION
Pretty prints double number into field_length characters buffer.
*/
after merge fixes
2004-08-19 03:02:09 +02:00
int Field_str::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
char buff[DOUBLE_TO_STRING_CONVERSION_BUFFER_SIZE];
take dec. point into account in store_double_in_string
2004-08-19 11:21:08 +02:00
uint length;
Portability fixes Fixed bug in end space handle for WHERE text_column="constant"
2004-08-26 17:26:38 +02:00
bool use_scientific_notation= TRUE;
ctype_utf8.result, ctype_utf8.test: adding test field.cc: bug#10714 Inserting double value into utf8 column crashes server: sprintf was executed with too big length, which caused crash on some Windows platforms.
2005-06-03 06:37:53 +02:00
uint char_length= field_length / charset()->mbmaxlen;
Fix conversion of floating point values to character fields when the absolute value of the float is less than 1, and also fix calculation of length for negative values. (Bug #7774)
2005-01-13 00:41:45 +01:00
/*
Check fabs(nr) against longest value that can be stored in field,
which depends on whether the value is < 1 or not, and negative or not
*/
double anr= fabs(nr);
int neg= (nr < 0.0) ? 1 : 0;
ctype_utf8.result, ctype_utf8.test: adding test field.cc: bug#10714 Inserting double value into utf8 column crashes server: sprintf was executed with too big length, which caused crash on some Windows platforms.
2005-06-03 06:37:53 +02:00
if (char_length > 4 && char_length < 32 &&
Merge
2005-06-03 06:44:13 +02:00
(anr < 1.0 ? anr > 1/(log_10[max(0,(int) char_length-neg-2)]) /* -2 for "0." */
ctype_utf8.result, ctype_utf8.test: adding test field.cc: bug#10714 Inserting double value into utf8 column crashes server: sprintf was executed with too big length, which caused crash on some Windows platforms.
2005-06-03 06:37:53 +02:00
: anr < log_10[char_length-neg]-1))
Portability fixes Fixed bug in end space handle for WHERE text_column="constant"
2004-08-26 17:26:38 +02:00
use_scientific_notation= FALSE;
Fix alignment.
2004-09-24 11:50:53 +02:00
Portability fixes Fixed bug in end space handle for WHERE text_column="constant"
2004-08-26 17:26:38 +02:00
length= (uint) my_sprintf(buff, (buff, "%-.*g",
(use_scientific_notation ?
ctype_utf8.result, ctype_utf8.test: adding test field.cc: bug#10714 Inserting double value into utf8 column crashes server: sprintf was executed with too big length, which caused crash on some Windows platforms.
2005-06-03 06:37:53 +02:00
max(0, (int)char_length-neg-5) :
char_length),
Portability fixes Fixed bug in end space handle for WHERE text_column="constant"
2004-08-26 17:26:38 +02:00
nr));
take dec. point into account in store_double_in_string
2004-08-19 11:21:08 +02:00
/*
+1 below is because "precision" in %g above means the
max. number of significant digits, not the output width.
Thus the width can be larger than number of significant digits by 1
(for decimal point)
ctype_utf8.result, ctype_utf8.test: adding test field.cc: bug#10714 Inserting double value into utf8 column crashes server: sprintf was executed with too big length, which caused crash on some Windows platforms.
2005-06-03 06:37:53 +02:00
the test for char_length < 5 is for extreme cases,
less strict assert to take into account weird cases
2004-08-19 23:10:33 +02:00
like inserting 500.0 in char(1)
take dec. point into account in store_double_in_string
2004-08-19 11:21:08 +02:00
*/
ctype_utf8.result, ctype_utf8.test: adding test field.cc: bug#10714 Inserting double value into utf8 column crashes server: sprintf was executed with too big length, which caused crash on some Windows platforms.
2005-06-03 06:37:53 +02:00
DBUG_ASSERT(char_length < 5 || length <= char_length+1);
A fix (Bug #5291: float -> string, truncation, sometimes without warning).
2004-09-03 16:38:16 +02:00
return store((const char *) buff, length, charset());
Bug 4937: different date -> string conversion when using SELECT ... UNION and INSERT ... SELECT ... UNION
2004-08-18 14:46:31 +02:00
}
Portability fixes Fixed bug in end space handle for WHERE text_column="constant"
2004-08-26 17:26:38 +02:00
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
uint Field::is_equal(create_field *new_field)
{
Fix for bug#18133: Adding index to unsigned int causes on-line add index not to be used
2006-04-03 13:25:28 +02:00
return (new_field->sql_type == real_type());
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
}
uint Field_str::is_equal(create_field *new_field)
{
if (((new_field->flags & (BINCMP_FLAG | BINARY_FLAG)) &&
!(flags & (BINCMP_FLAG | BINARY_FLAG))) ||
(!(new_field->flags & (BINCMP_FLAG | BINARY_FLAG)) &&
(flags & (BINCMP_FLAG | BINARY_FLAG))))
return 0; /* One of the fields is binary and the other one isn't */
Fix for bug#18133: Adding index to unsigned int causes on-line add index not to be used
2006-04-03 13:25:28 +02:00
return ((new_field->sql_type == real_type()) &&
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
new_field->charset == field_charset &&
new_field->length == max_length());
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_string::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
Use of new num->str functions fix in ucs2 charset
2002-12-11 12:29:23 +01:00
char buff[64];
int l;
CHARSET_INFO *cs=charset();
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
l= (cs->cset->longlong10_to_str)(cs,buff,sizeof(buff),
unsigned_val ? 10 : -10, nr);
Use of new num->str functions fix in ucs2 charset
2002-12-11 12:29:23 +01:00
return Field_string::store(buff,(uint)l,cs);
Import changeset
2000-07-31 21:29:14 +02:00
}
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
int Field_longstr::store_decimal(const my_decimal *d)
{
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
char buff[DECIMAL_MAX_STR_LENGTH+1];
String str(buff, sizeof(buff), &my_charset_bin);
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal2string(E_DEC_FATAL_ERROR, d, 0, 0, 0, &str);
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
return store(str.ptr(), str.length(), str.charset());
Precision Math implementation
2005-02-08 23:50:45 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
double Field_string::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
char *end_not_used;
CHARSET_INFO *cs= charset();
return my_strntod(cs,ptr,field_length,&end_not_used,&not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
longlong Field_string::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
char *end_not_used;
Use of new string->number conversion routines
2002-12-09 11:08:42 +01:00
CHARSET_INFO *cs=charset();
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
return my_strntoll(cs,ptr,field_length,10,&end_not_used,&not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
String *Field_string::val_str(String *val_buffer __attribute__((unused)),
String *val_ptr)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Bug fix: http://bugs.mysql.com/bug.php?id=1264
2003-09-16 12:43:17 +02:00
uint length= field_charset->cset->lengthsp(field_charset, ptr, field_length);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
/* See the comment for Field_long::store(long long) */
DBUG_ASSERT(table->in_use == current_thd);
Bug fix: http://bugs.mysql.com/bug.php?id=1264
2003-09-16 12:43:17 +02:00
val_ptr->set((const char*) ptr, length, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
return val_ptr;
}
cast.result, cast.test: Added test case for bug #11283. field.h, field.cc: Fixed bug #11283: wrong conversion from varchar to decimal. Added methods Field_string::val_decimal, Field_varstring::val_decimal, Field_blob::val_decimal. They are not inherited from the base class Field_longstr anymore.
2005-06-15 17:27:33 +02:00
my_decimal *Field_string::val_decimal(my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
cast.result, cast.test: Added test case for bug #11283. field.h, field.cc: Fixed bug #11283: wrong conversion from varchar to decimal. Added methods Field_string::val_decimal, Field_varstring::val_decimal, Field_blob::val_decimal. They are not inherited from the base class Field_longstr anymore.
2005-06-15 17:27:33 +02:00
str2my_decimal(E_DEC_FATAL_ERROR, ptr, field_length, charset(),
decimal_value);
return decimal_value;
}
Import changeset
2000-07-31 21:29:14 +02:00
int Field_string::cmp(const char *a_ptr, const char *b_ptr)
{
Simple optimization
2004-11-09 02:20:58 +01:00
uint a_len, b_len;
After merge fixes Added initialization of all important global variables
2003-05-21 20:39:58 +02:00
if (field_charset->strxfrm_multiply > 1)
Fixed problem when comparing a key for a multi-byte-character set. (bug 152) Use 0x.... as strings if 'new' mode. (bug 152) Don't report -max on windows when InnoDB is enabled. (bug 332) Reset current_linfo; This could cause a hang when doing PURGE LOGS. Fix for row numbers in EXPLAIN (bug 322) Fix that USE_FRM works for all table types (bug 97)
2003-04-27 21:12:08 +02:00
{
/*
We have to remove end space to be able to compare multi-byte-characters
like in latin_de 'ae' and 0xe4
*/
CHARSET_INFO structure reorganization for easier maintainance
2003-05-23 14:45:52 +02:00
return field_charset->coll->strnncollsp(field_charset,
key_cmp -> key_cmp_if_same New records_in_range() interface (similar to read_range()) Macros for faster bitmap handling Simplify read_range() code (#WL1786) New general key_cmp() function to compare keys
2004-05-16 13:48:32 +02:00
(const uchar*) a_ptr, field_length,
(const uchar*) b_ptr,
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
field_length, 0);
Fixed problem when comparing a key for a multi-byte-character set. (bug 152) Use 0x.... as strings if 'new' mode. (bug 152) Don't report -max on windows when InnoDB is enabled. (bug 332) Reset current_linfo; This could cause a hang when doing PURGE LOGS. Fix for row numbers in EXPLAIN (bug 322) Fix that USE_FRM works for all table types (bug 97)
2003-04-27 21:12:08 +02:00
}
Simple optimization
2004-11-09 02:20:58 +01:00
if (field_charset->mbmaxlen != 1)
{
uint char_len= field_length/field_charset->mbmaxlen;
a_len= my_charpos(field_charset, a_ptr, a_ptr + field_length, char_len);
b_len= my_charpos(field_charset, b_ptr, b_ptr + field_length, char_len);
}
else
a_len= b_len= field_length;
ctype_ujis.result, ctype_ujis.test, field.cc: Bug#6345 Unexpected behaviour with partial indices
2004-11-08 10:15:01 +01:00
return my_strnncoll(field_charset,(const uchar*) a_ptr, a_len,
(const uchar*) b_ptr, b_len);
Import changeset
2000-07-31 21:29:14 +02:00
}
ctype_ujis.result, ctype_ujis.test, field.cc: Bug#6345 Unexpected behaviour with partial indices
2004-11-08 10:15:01 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
void Field_string::sort_string(char *to,uint length)
{
strnxfrm() for simple and binary charsets fixes
2003-01-16 12:20:38 +01:00
uint tmp=my_strnxfrm(field_charset,
BTREE-indexes in HEAP tables can now be used to optimize ORDER BY Don't read character set files if we are using only the default charset. In most cases the user will not anymore get a warning about missing character set files Compare strings with space extend instead of space strip. Now the following comparisons holds: "a" == "a " and "a\t" < "a". (Bug #3152). Note: Because of the above fix, one has to do a REPAIR on any table that has an ascii character < 32 last in a CHAR/VARCHAR/TEXT columns.
2004-03-25 14:05:01 +01:00
(unsigned char *) to, length,
(unsigned char *) ptr, field_length);
bug#7284: strnxfrm returns different results for equal strings
2005-01-13 15:12:04 +01:00
DBUG_ASSERT(tmp == length);
Import changeset
2000-07-31 21:29:14 +02:00
}
void Field_string::sql_type(String &res) const
{
field.cc, sql_mode.result, sql_mode.test: "SHOW CREATE TABLE" mysql-4.0 and mysql-3.23 compatibiliry mode change: Check that a binary collation adds 'binary' suffix into a char() column definition in mysql40 and mysql2323 modes. This allows not to lose the column's case sensitivity when loading the dump in pre-4.1 servers.
2004-11-04 05:50:07 +01:00
THD *thd= table->in_use;
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
ulong length;
length= cs->cset->snprintf(cs,(char*) res.ptr(),
res.alloced_length(), "%s(%d)",
((type() == MYSQL_TYPE_VAR_STRING &&
!thd->variables.new_mode) ?
(has_charset() ? "varchar" : "varbinary") :
1. New data types, from the user point of view: BINARY(N) and VARBIBARY(N) 2. More 4.0 compatibility and more BINARY keyword consistency: 2a. CREATE TABLE a (a CHAR(N) BINARY) is now synonym for CREATE TABLE a (a CHAR(N) COLLATE xxxx_bin) 2b. SELECT BINARY x is still synonin for SELECT x COLLATE xxxxx_bin.
2004-03-26 13:11:46 +01:00
(has_charset() ? "char" : "binary")),
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
(int) field_length / charset()->mbmaxlen);
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
res.length(length);
field.cc, sql_mode.result, sql_mode.test: "SHOW CREATE TABLE" mysql-4.0 and mysql-3.23 compatibiliry mode change: Check that a binary collation adds 'binary' suffix into a char() column definition in mysql40 and mysql2323 modes. This allows not to lose the column's case sensitivity when loading the dump in pre-4.1 servers.
2004-11-04 05:50:07 +01:00
if ((thd->variables.sql_mode & (MODE_MYSQL323 | MODE_MYSQL40)) &&
has_charset() && (charset()->state & MY_CS_BINSORT))
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN(" binary"));
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
char *Field_string::pack(char *to, const char *from, uint max_length)
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
{
Portability fixes Fixed bug in end space handle for WHERE text_column="constant"
2004-08-26 17:26:38 +02:00
uint length= min(field_length,max_length);
uint char_length= max_length/field_charset->mbmaxlen;
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
if (length > char_length)
followup
2004-08-24 18:20:27 +02:00
char_length= my_charpos(field_charset, from, from+length, char_length);
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
set_if_smaller(length, char_length);
followup
2004-08-24 18:20:27 +02:00
while (length && from[length-1] == ' ')
length--;
Review of all code pushed since last review Simple optimzations and cleanups Removed compiler warnings and fixed portability issues Added client functions 'mysql_embedded()' to allow client to check if we are using embedded server Fixes for purify
2004-10-20 00:28:42 +02:00
*to++= (char) (uchar) length;
bug #5723 length(varchar utf8 field) returns verying results
2004-09-26 18:52:20 +02:00
if (field_length > 255)
Review of all code pushed since last review Simple optimzations and cleanups Removed compiler warnings and fixed portability issues Added client functions 'mysql_embedded()' to allow client to check if we are using embedded server Fixes for purify
2004-10-20 00:28:42 +02:00
*to++= (char) (uchar) (length >> 8);
bug #5723 length(varchar utf8 field) returns verying results
2004-09-26 18:52:20 +02:00
memcpy(to, from, length);
return to+length;
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
const char *Field_string::unpack(char *to, const char *from)
{
bug #5723 length(varchar utf8 field) returns verying results
2004-09-26 18:52:20 +02:00
uint length;
if (field_length > 255)
{
length= uint2korr(from);
from+= 2;
}
else
length= (uint) (uchar) *from++;
Import changeset
2000-07-31 21:29:14 +02:00
memcpy(to, from, (int) length);
bfill(to+length, field_length - length, ' ');
return from+length;
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
/*
Compare two packed keys
SYNOPSIS
pack_cmp()
a New key
b Original key
length Key length
insert_or_update 1 if this is an insert or update
RETURN
< 0 a < b
0 a = b
> 0 a > b
*/
int Field_string::pack_cmp(const char *a, const char *b, uint length,
my_bool insert_or_update)
Import changeset
2000-07-31 21:29:14 +02:00
{
bug #5723 length(varchar utf8 field) returns verying results
2004-09-26 18:52:20 +02:00
uint a_length, b_length;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
if (length > 255)
bug #5723 length(varchar utf8 field) returns verying results
2004-09-26 18:52:20 +02:00
{
a_length= uint2korr(a);
b_length= uint2korr(b);
a+= 2;
b+= 2;
}
else
{
a_length= (uint) (uchar) *a++;
b_length= (uint) (uchar) *b++;
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) a, a_length,
(const uchar*) b, b_length,
insert_or_update);
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
/*
Compare a packed key against row
SYNOPSIS
pack_cmp()
key Original key
length Key length. (May be less than field length)
insert_or_update 1 if this is an insert or update
RETURN
< 0 row < key
0 row = key
> 0 row > key
*/
int Field_string::pack_cmp(const char *key, uint length,
my_bool insert_or_update)
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
{
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
uint row_length, key_length;
char *end;
if (length > 255)
bug #5723 length(varchar utf8 field) returns verying results
2004-09-26 18:52:20 +02:00
{
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
key_length= uint2korr(key);
key+= 2;
bug #5723 length(varchar utf8 field) returns verying results
2004-09-26 18:52:20 +02:00
}
else
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
key_length= (uint) (uchar) *key++;
/* Only use 'length' of key, not field_length */
end= ptr + length;
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
while (end > ptr && end[-1] == ' ')
end--;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
row_length= (uint) (end - ptr);
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) ptr, row_length,
(const uchar*) key, key_length,
insert_or_update);
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
}
Portability fixes for Fortre C++ 5.0 (on Sun) in 32 and 64 bit modes.
2002-11-07 02:54:00 +01:00
uint Field_string::packed_col_length(const char *data_ptr, uint length)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fix that mysqladmin shutdown can be interrupted with ^C Fixed bug with BLOB keys in BDB tables Fixed problem with MERGE tables on OS with 32 bit files Fixed that TIME_TO_SEC() works with negative times
2001-11-03 14:18:09 +01:00
if (length > 255)
Portability fixes for Fortre C++ 5.0 (on Sun) in 32 and 64 bit modes.
2002-11-07 02:54:00 +01:00
return uint2korr(data_ptr)+2;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return (uint) ((uchar) *data_ptr)+1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
uint Field_string::max_packed_col_length(uint max_length)
{
Fix that mysqladmin shutdown can be interrupted with ^C Fixed bug with BLOB keys in BDB tables Fixed problem with MERGE tables on OS with 32 bit files Fixed that TIME_TO_SEC() works with negative times
2001-11-03 14:18:09 +01:00
return (max_length > 255 ? 2 : 1)+max_length;
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Field *Field_string::new_field(MEM_ROOT *root, struct st_table *new_table)
{
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
Field *field;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
if (type() != MYSQL_TYPE_VAR_STRING || table == new_table)
return Field::new_field(root, new_table);
/*
Old VARCHAR field which should be modified to a VARCHAR on copy
This is done to ensure that ALTER TABLE will convert old VARCHAR fields
to now VARCHAR fields.
*/
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
if ((field= new Field_varstring(field_length, maybe_null(), field_name,
new_table->s, charset())))
Bug#13707 - Server crash with INSERT DELAYED on MyISAM table After push fix. Must not access new_field if it is NULL.
2005-11-14 19:53:41 +01:00
{
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
field->init(new_table);
Bug#13707 - Server crash with INSERT DELAYED on MyISAM table After push fix. Must not access new_field if it is NULL.
2005-11-14 19:53:41 +01:00
/*
delayed_insert::get_local_table() needs a ptr copied from old table.
This is what other new_field() methods do too. The above method of
Field_varstring sets ptr to NULL.
*/
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
field->ptr= ptr;
field->null_ptr= null_ptr;
field->null_bit= null_bit;
Bug#13707 - Server crash with INSERT DELAYED on MyISAM table After push fix. Must not access new_field if it is NULL.
2005-11-14 19:53:41 +01:00
}
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
return field;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
}
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
VARCHAR type
Data in field->ptr is stored as:
1 or 2 bytes length-prefix-header (from Field_varstring::length_bytes)
data
NOTE:
When VARCHAR is stored in a key (for handler::index_read() etc) it's always
stored with a 2 byte prefix. (Just like blob keys).
Normally length_bytes is calculated as (field_length < 256 : 1 ? 2)
The exception is if there is a prefix key field that is part of a long
VARCHAR, in which case field_length for this may be 1 but the length_bytes
is 2.
Import changeset
2000-07-31 21:29:14 +02:00
****************************************************************************/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_varstring::store(const char *from,uint length,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
uint32 not_used, copy_length;
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
String tmpstr(buff,sizeof(buff), &my_charset_bin);
manually merged Gluh's SESSION/GLOBAL for @variables fix in sql_yacc.yy and Bar's well_formed_len() changes in ndb code did not make it and should be re-applied manually
2005-04-06 21:19:20 +02:00
int error_code= 0, well_formed_error;
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
enum MYSQL_ERROR::enum_warning_level level= MYSQL_ERROR::WARN_LEVEL_WARN;
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
/* Convert character set if necessary */
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
if (String::needs_conversion(length, cs, field_charset, &not_used))
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
{
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
uint conv_errors;
tmpstr.copy(from, length, cs, field_charset, &conv_errors);
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
from= tmpstr.ptr();
length= tmpstr.length();
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
if (conv_errors)
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
error_code= WARN_DATA_TRUNCATED;
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
/*
Make sure we don't break a multibyte sequence
as well as don't copy a malformed data.
*/
copy_length= field_charset->cset->well_formed_len(field_charset,
from,from+length,
field_length/
manually merged Gluh's SESSION/GLOBAL for @variables fix in sql_yacc.yy and Bar's well_formed_len() changes in ndb code did not make it and should be re-applied manually
2005-04-06 21:19:20 +02:00
field_charset->mbmaxlen,
&well_formed_error);
Fix for BUG#15588: String overrun during sp-vars.test The bug appears after implementation of WL#2984 (Make stored routine variables work according to the standard).
2006-01-20 13:59:22 +01:00
memmove(ptr + length_bytes, from, copy_length);
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
if (length_bytes == 1)
*ptr= (uchar) copy_length;
else
int2store(ptr, copy_length);
If VARCHAR strips only trailing spaces, then produce a note, not a warning or error.
2005-01-11 13:49:45 +01:00
// Check if we lost something other than just trailing spaces
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
if ((copy_length < length) && table->in_use->count_cuted_fields &&
!error_code)
If VARCHAR strips only trailing spaces, then produce a note, not a warning or error.
2005-01-11 13:49:45 +01:00
{
Report truncation of spaces when inserting into a BINARY or VARBINARY field. (Bug #14299)
2005-11-16 04:21:05 +01:00
if (!binary())
{
const char *end= from + length;
from+= copy_length;
from+= field_charset->cset->scan(field_charset, from, end, MY_SEQ_SPACES);
/* If we lost only spaces then produce a NOTE, not a WARNING */
if (from == end)
level= MYSQL_ERROR::WARN_LEVEL_NOTE;
}
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
error_code= WARN_DATA_TRUNCATED;
If VARCHAR strips only trailing spaces, then produce a note, not a warning or error.
2005-01-11 13:49:45 +01:00
}
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
if (error_code)
a fix (bug #9029 Traditional: Wrong SQLSTATE returned for string truncation).
2005-03-10 14:08:45 +01:00
{
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
if (level == MYSQL_ERROR::WARN_LEVEL_WARN &&
table->in_use->abort_on_warning)
error_code= ER_DATA_TOO_LONG;
set_warning(level, error_code, 1);
Manual merge
2005-06-23 20:22:30 +02:00
return 2;
a fix (bug #9029 Traditional: Wrong SQLSTATE returned for string truncation).
2005-03-10 14:08:45 +01:00
}
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_varstring::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
Use of new num->str functions fix in ucs2 charset
2002-12-11 12:29:23 +01:00
char buff[64];
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint length;
length= (uint) (field_charset->cset->longlong10_to_str)(field_charset,
buff,
sizeof(buff),
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
(unsigned_val ? 10:
-10),
nr);
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
return Field_varstring::store(buff, length, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_varstring::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
char *end_not_used;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
return my_strntod(field_charset, ptr+length_bytes, length, &end_not_used,
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
&not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
longlong Field_varstring::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
char *end_not_used;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
return my_strntoll(field_charset, ptr+length_bytes, length, 10,
&end_not_used, &not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
String *Field_varstring::val_str(String *val_buffer __attribute__((unused)),
String *val_ptr)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
val_ptr->set((const char*) ptr+length_bytes, length, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
return val_ptr;
}
cast.result, cast.test: Added test case for bug #11283. field.h, field.cc: Fixed bug #11283: wrong conversion from varchar to decimal. Added methods Field_string::val_decimal, Field_varstring::val_decimal, Field_blob::val_decimal. They are not inherited from the base class Field_longstr anymore.
2005-06-15 17:27:33 +02:00
my_decimal *Field_varstring::val_decimal(my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
cast.result, cast.test: Added test case for bug #11283. field.h, field.cc: Fixed bug #11283: wrong conversion from varchar to decimal. Added methods Field_string::val_decimal, Field_varstring::val_decimal, Field_blob::val_decimal. They are not inherited from the base class Field_longstr anymore.
2005-06-15 17:27:33 +02:00
uint length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
str2my_decimal(E_DEC_FATAL_ERROR, ptr+length_bytes, length, charset(),
decimal_value);
return decimal_value;
}
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
int Field_varstring::cmp_max(const char *a_ptr, const char *b_ptr,
uint max_len)
Import changeset
2000-07-31 21:29:14 +02:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint a_length, b_length;
Import changeset
2000-07-31 21:29:14 +02:00
int diff;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
if (length_bytes == 1)
{
a_length= (uint) (uchar) *a_ptr;
b_length= (uint) (uchar) *b_ptr;
}
else
{
a_length= uint2korr(a_ptr);
b_length= uint2korr(b_ptr);
}
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
set_if_smaller(a_length, max_len);
set_if_smaller(b_length, max_len);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
diff= field_charset->coll->strnncollsp(field_charset,
(const uchar*) a_ptr+
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
length_bytes,
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
a_length,
(const uchar*) b_ptr+
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
length_bytes,
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
b_length,0);
return diff;
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
/*
NOTE: varstring and blob keys are ALWAYS stored with a 2 byte length prefix
*/
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int Field_varstring::key_cmp(const byte *key_ptr, uint max_key_length)
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
uint char_length= max_key_length / field_charset->mbmaxlen;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
char_length= my_charpos(field_charset, ptr + length_bytes,
ptr + length_bytes + length, char_length);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
set_if_smaller(length, char_length);
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) ptr + length_bytes,
length,
(const uchar*) key_ptr+
HA_KEY_BLOB_LENGTH,
uint2korr(key_ptr), 0);
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
/*
Compare to key segments (always 2 byte length prefix)
NOTE
This is used only to compare key segments created for index_read().
(keys are created and compared in key.cc)
*/
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int Field_varstring::key_cmp(const byte *a,const byte *b)
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) a +
HA_KEY_BLOB_LENGTH,
uint2korr(a),
(const uchar*) b +
HA_KEY_BLOB_LENGTH,
uint2korr(b),
0);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
void Field_varstring::sort_string(char *to,uint length)
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint tot_length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
Move handling of suffix_length from strnxfrm_bin() to filesort to ensure proper sorting of all kind of binary objects field::sort_key() now adds length last for varbinary/blob VARBINARY/BLOB is now sorted by filesort so that shorter strings comes before longer ones Fixed issues in test cases from last merge
2005-10-13 23:04:52 +02:00
if (field_charset == &my_charset_bin)
{
/* Store length last in high-byte order to sort longer strings first */
if (length_bytes == 1)
to[length-1]= tot_length;
else
mi_int2store(to+length-2, tot_length);
length-= length_bytes;
}
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
tot_length= my_strnxfrm(field_charset,
(uchar*) to, length,
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
(uchar*) ptr + length_bytes,
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
tot_length);
Merge with 4.1
2005-01-15 13:09:45 +01:00
DBUG_ASSERT(tot_length == length);
Import changeset
2000-07-31 21:29:14 +02:00
}
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
enum ha_base_keytype Field_varstring::key_type() const
{
enum ha_base_keytype res;
if (binary())
res= length_bytes == 1 ? HA_KEYTYPE_VARBINARY1 : HA_KEYTYPE_VARBINARY2;
else
res= length_bytes == 1 ? HA_KEYTYPE_VARTEXT1 : HA_KEYTYPE_VARTEXT2;
return res;
}
Import changeset
2000-07-31 21:29:14 +02:00
void Field_varstring::sql_type(String &res) const
{
- VARCHAR(n) with binary character set is now displayed as VARBINARY(N). - CREATE TABLE t1 SELECT BINARY 'literal' now creates a VARBINARY() column, not a BINARY().
2004-12-08 11:03:29 +01:00
THD *thd= table->in_use;
sql_type() is now ucs2 compatible
2002-12-17 16:57:22 +01:00
CHARSET_INFO *cs=res.charset();
- VARCHAR(n) with binary character set is now displayed as VARBINARY(N). - CREATE TABLE t1 SELECT BINARY 'literal' now creates a VARBINARY() column, not a BINARY().
2004-12-08 11:03:29 +01:00
ulong length;
length= cs->cset->snprintf(cs,(char*) res.ptr(),
res.alloced_length(), "%s(%d)",
(has_charset() ? "varchar" : "varbinary"),
(int) field_length / charset()->mbmaxlen);
Some fixes to get closer to UCS2 compatible SHOW FIELDS
2002-12-17 11:35:19 +01:00
res.length(length);
- VARCHAR(n) with binary character set is now displayed as VARBINARY(N). - CREATE TABLE t1 SELECT BINARY 'literal' now creates a VARBINARY() column, not a BINARY().
2004-12-08 11:03:29 +01:00
if ((thd->variables.sql_mode & (MODE_MYSQL323 | MODE_MYSQL40)) &&
has_charset() && (charset()->state & MY_CS_BINSORT))
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN(" binary"));
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
/*
Functions to create a packed row.
Here the number of length bytes are depending on the given max_length
*/
Import changeset
2000-07-31 21:29:14 +02:00
char *Field_varstring::pack(char *to, const char *from, uint max_length)
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint length= length_bytes == 1 ? (uint) (uchar) *from : uint2korr(from);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
set_if_smaller(max_length, field_length);
Import changeset
2000-07-31 21:29:14 +02:00
if (length > max_length)
length=max_length;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*to++= (char) (length & 255);
Import changeset
2000-07-31 21:29:14 +02:00
if (max_length > 255)
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*to++= (char) (length >> 8);
Import changeset
2000-07-31 21:29:14 +02:00
if (length)
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
memcpy(to, from+length_bytes, length);
Import changeset
2000-07-31 21:29:14 +02:00
return to+length;
}
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
char *Field_varstring::pack_key(char *to, const char *key, uint max_length)
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint length= length_bytes == 1 ? (uint) (uchar) *key : uint2korr(key);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
uint char_length= ((field_charset->mbmaxlen > 1) ?
max_length/field_charset->mbmaxlen : max_length);
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
key+= length_bytes;
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
if (length > char_length)
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
char_length= my_charpos(field_charset, key, key+length, char_length);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
set_if_smaller(length, char_length);
}
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
*to++= (char) (length & 255);
if (max_length > 255)
*to++= (char) (length >> 8);
if (length)
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
memcpy(to, key, length);
return to+length;
}
/*
Unpack a key into a record buffer.
SYNOPSIS
unpack_key()
to Pointer into the record buffer.
key Pointer to the packed key.
max_length Key length limit from key description.
DESCRIPTION
A VARCHAR key has a maximum size of 64K-1.
In its packed form, the length field is one or two bytes long,
depending on 'max_length'.
RETURN
Pointer to end of 'key' (To the next key part if multi-segment key)
*/
const char *Field_varstring::unpack_key(char *to, const char *key,
uint max_length)
{
/* get length of the blob key */
uint32 length= *((uchar*) key++);
if (max_length > 255)
length+= (*((uchar*) key++)) << 8;
/* put the length into the record buffer */
if (length_bytes == 1)
*ptr= (uchar) length;
else
int2store(ptr, length);
memcpy(ptr + length_bytes, key, length);
return key + length;
}
/*
Create a packed key that will be used for storage in the index tree
SYNOPSIS
pack_key_from_key_image()
to Store packed key segment here
from Key segment (as given to index_read())
max_length Max length of key
RETURN
end of key storage
*/
char *Field_varstring::pack_key_from_key_image(char *to, const char *from,
uint max_length)
{
/* Key length is always stored as 2 bytes */
uint length= uint2korr(from);
if (length > max_length)
length= max_length;
*to++= (char) (length & 255);
if (max_length > 255)
*to++= (char) (length >> 8);
if (length)
memcpy(to, from+HA_KEY_BLOB_LENGTH, length);
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
return to+length;
}
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
/*
unpack field packed with Field_varstring::pack()
*/
Import changeset
2000-07-31 21:29:14 +02:00
const char *Field_varstring::unpack(char *to, const char *from)
{
uint length;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
if (length_bytes == 1)
Import changeset
2000-07-31 21:29:14 +02:00
length= (uint) (uchar) (*to= *from++);
else
{
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
length= uint2korr(from);
to[0]= *from++;
to[1]= *from++;
Import changeset
2000-07-31 21:29:14 +02:00
}
if (length)
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
memcpy(to+ length_bytes, from, length);
Import changeset
2000-07-31 21:29:14 +02:00
return from+length;
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int Field_varstring::pack_cmp(const char *a, const char *b, uint key_length,
my_bool insert_or_update)
Import changeset
2000-07-31 21:29:14 +02:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint a_length, b_length;
Import changeset
2000-07-31 21:29:14 +02:00
if (key_length > 255)
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
a_length=uint2korr(a); a+= 2;
b_length=uint2korr(b); b+= 2;
Import changeset
2000-07-31 21:29:14 +02:00
}
else
{
a_length= (uint) (uchar) *a++;
b_length= (uint) (uchar) *b++;
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) a, a_length,
(const uchar*) b, b_length,
insert_or_update);
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int Field_varstring::pack_cmp(const char *b, uint key_length,
my_bool insert_or_update)
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
char *a= ptr+ length_bytes;
uint a_length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
uint b_length;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
uint char_length= ((field_charset->mbmaxlen > 1) ?
key_length / field_charset->mbmaxlen : key_length);
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
if (key_length > 255)
{
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
b_length=uint2korr(b); b+= HA_KEY_BLOB_LENGTH;
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
}
else
b_length= (uint) (uchar) *b++;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
if (a_length > char_length)
{
char_length= my_charpos(field_charset, a, a+a_length, char_length);
set_if_smaller(a_length, char_length);
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) a,
a_length,
(const uchar*) b, b_length,
insert_or_update);
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Portability fixes for Fortre C++ 5.0 (on Sun) in 32 and 64 bit modes.
2002-11-07 02:54:00 +01:00
uint Field_varstring::packed_col_length(const char *data_ptr, uint length)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fix that mysqladmin shutdown can be interrupted with ^C Fixed bug with BLOB keys in BDB tables Fixed problem with MERGE tables on OS with 32 bit files Fixed that TIME_TO_SEC() works with negative times
2001-11-03 14:18:09 +01:00
if (length > 255)
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
return uint2korr(data_ptr)+2;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return (uint) ((uchar) *data_ptr)+1;
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
uint Field_varstring::max_packed_col_length(uint max_length)
{
Fix that mysqladmin shutdown can be interrupted with ^C Fixed bug with BLOB keys in BDB tables Fixed problem with MERGE tables on OS with 32 bit files Fixed that TIME_TO_SEC() works with negative times
2001-11-03 14:18:09 +01:00
return (max_length > 255 ? 2 : 1)+max_length;
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
void Field_varstring::get_key_image(char *buff, uint length, imagetype type)
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint f_length= length_bytes == 1 ? (uint) (uchar) *ptr : uint2korr(ptr);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
uint char_length= length / field_charset->mbmaxlen;
BUG#9339 - Updating a column does not work if the table has a UTF-8 VARCHAR primary key fixed my_charpos() call in Field_varstring::get_key_image
2005-03-22 20:13:49 +01:00
char *pos= ptr+length_bytes;
char_length= my_charpos(field_charset, pos, pos + f_length, char_length);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
set_if_smaller(f_length, char_length);
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
/* Key is always stored with 2 bytes */
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int2store(buff,f_length);
BUG#9339 - Updating a column does not work if the table has a UTF-8 VARCHAR primary key fixed my_charpos() call in Field_varstring::get_key_image
2005-03-22 20:13:49 +01:00
memcpy(buff+HA_KEY_BLOB_LENGTH, pos, f_length);
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
if (f_length < length)
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
{
/*
Must clear this as we do a memcmp in opt_range.cc to detect
identical keys
*/
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
bzero(buff+HA_KEY_BLOB_LENGTH+f_length, (length-f_length));
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
void Field_varstring::set_key_image(char *buff,uint length)
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
length= uint2korr(buff); // Real length is here
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
(void) Field_varstring::store(buff+HA_KEY_BLOB_LENGTH, length,
field_charset);
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int Field_varstring::cmp_binary(const char *a_ptr, const char *b_ptr,
uint32 max_length)
{
uint32 a_length,b_length;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
if (length_bytes == 1)
{
a_length= (uint) (uchar) *a_ptr;
b_length= (uint) (uchar) *b_ptr;
}
else
{
a_length= uint2korr(a_ptr);
b_length= uint2korr(b_ptr);
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
set_if_smaller(a_length, max_length);
set_if_smaller(b_length, max_length);
if (a_length != b_length)
return 1;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
return memcmp(a_ptr+length_bytes, b_ptr+length_bytes, a_length);
}
Field *Field_varstring::new_field(MEM_ROOT *root, struct st_table *new_table)
{
Field_varstring *res= (Field_varstring*) Field::new_field(root, new_table);
if (res)
res->length_bytes= length_bytes;
return res;
}
Field *Field_varstring::new_key_field(MEM_ROOT *root,
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
struct st_table *new_table,
char *new_ptr, uchar *new_null_ptr,
uint new_null_bit)
{
Field_varstring *res;
if ((res= (Field_varstring*) Field::new_key_field(root,
new_table,
new_ptr,
new_null_ptr,
new_null_bit)))
{
/* Keys length prefixes are always packed with 2 bytes */
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
res->length_bytes= 2;
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
}
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
return res;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
}
rename of net_pkg.cc to protocol.cc Class for sending data from server to client (Protocol) This handles both the old ( <= 4.0 ) protocol and then new binary protocol that is used for prepared statements.
2002-12-11 08:17:51 +01:00
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
uint Field_varstring::is_equal(create_field *new_field)
{
Fix for bug#18133: Adding index to unsigned int causes on-line add index not to be used
2006-04-03 13:25:28 +02:00
if (new_field->sql_type == real_type() &&
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
new_field->charset == field_charset)
{
if (new_field->length == max_length())
return IS_EQUAL_YES;
if (new_field->length > max_length() &&
((new_field->length <= 255 && max_length() <= 255) ||
(new_field->length > 255 && max_length() > 255)))
return IS_EQUAL_PACK_LENGTH; // VARCHAR, longer variable length
}
return IS_EQUAL_NO;
}
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
** blob type
** A blob is saved as a length and a pointer. The length is stored in the
** packlength slot and may be from 1-4.
****************************************************************************/
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
Field_blob::Field_blob(char *ptr_arg, uchar *null_ptr_arg, uchar null_bit_arg,
Import changeset
2000-07-31 21:29:14 +02:00
enum utype unireg_check_arg, const char *field_name_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
TABLE_SHARE *share, uint blob_pack_length,
BINARY charset is now used instead of binary_flag
2002-10-25 10:58:32 +02:00
CHARSET_INFO *cs)
Merge mysql.com:/opt/local/work/mysql-4.1-root into mysql.com:/opt/local/work/mysql-5.0-root
2005-07-14 18:02:32 +02:00
:Field_longstr(ptr_arg, BLOB_PACK_LENGTH_TO_MAX_LENGH(blob_pack_length),
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
null_ptr_arg, null_bit_arg, unireg_check_arg, field_name_arg,
cs),
SCRUM: WL#1163 (Making spatial code optional) Pack of changes to do in sql/ code.
2004-01-15 18:06:22 +01:00
packlength(blob_pack_length)
Import changeset
2000-07-31 21:29:14 +02:00
{
flags|= BLOB_FLAG;
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
share->blob_fields++;
Merge mysql.com:/home/alik/MySQL/devel/5.0-wl2984 into mysql.com:/home/alik/MySQL/devel/5.1-wl2984-merge
2005-12-09 13:14:19 +01:00
/* TODO: why do not fill table->s->blob_field array here? */
Import changeset
2000-07-31 21:29:14 +02:00
}
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
void Field_blob::store_length(uint32 number)
Import changeset
2000-07-31 21:29:14 +02:00
{
switch (packlength) {
case 1:
ptr[0]= (uchar) number;
break;
case 2:
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int2store(ptr,(unsigned short) number);
}
else
#endif
shortstore(ptr,(unsigned short) number);
break;
case 3:
int3store(ptr,number);
break;
case 4:
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,number);
}
else
#endif
longstore(ptr,number);
}
}
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 Field_blob::get_length(const char *pos)
Import changeset
2000-07-31 21:29:14 +02:00
{
switch (packlength) {
case 1:
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
return (uint32) (uchar) pos[0];
Import changeset
2000-07-31 21:29:14 +02:00
case 2:
{
uint16 tmp;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
tmp=sint2korr(pos);
else
#endif
shortget(tmp,pos);
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
return (uint32) tmp;
Import changeset
2000-07-31 21:29:14 +02:00
}
case 3:
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
return (uint32) uint3korr(pos);
Import changeset
2000-07-31 21:29:14 +02:00
case 4:
{
uint32 tmp;
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
tmp=uint4korr(pos);
else
#endif
longget(tmp,pos);
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
return (uint32) tmp;
Import changeset
2000-07-31 21:29:14 +02:00
}
}
return 0; // Impossible
}
bug#2686 - index_merge select on BerkeleyDB table with varchar PK causes mysqld to crash. Added put_length() to get_length() and unpack_key() to pack_key(). Keys were packed with the minimum size of the length field for the key part and unpacked with length size of the base column. For the purpose of optimal key packing we have the method pack_key(), while rows are packed with pack(). Now keys are unpacked with unpack_key() and no longer with unpack() which is used for rows.
2004-06-24 14:54:28 +02:00
/*
Put a blob length field into a record buffer.
SYNOPSIS
Field_blob::put_length()
pos Pointer into the record buffer.
length The length value to put.
DESCRIPTION
Depending on the maximum length of a blob, its length field is
put into 1 to 4 bytes. This is a property of the blob object,
described by 'packlength'.
RETURN
nothing
*/
void Field_blob::put_length(char *pos, uint32 length)
{
switch (packlength) {
case 1:
*pos= (char) length;
break;
case 2:
int2store(pos, length);
break;
case 3:
int3store(pos, length);
break;
case 4:
int4store(pos, length);
break;
}
}
field.cc: rename len -> length, to look in common style
2003-02-27 08:44:45 +01:00
int Field_blob::store(const char *from,uint length,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Minor clean-ups for the previous commit.
2005-04-06 12:12:44 +02:00
int error= 0, well_formed_error;
field.cc: rename len -> length, to look in common style
2003-02-27 08:44:45 +01:00
if (!length)
Import changeset
2000-07-31 21:29:14 +02:00
{
bzero(ptr,Field_blob::pack_length());
}
else
{
field.cc: http://bugs.mysql.com/bug.php?id=2218 updating utf-8 text field generate nonsense chars Fix for the above bug.
2004-01-15 12:58:06 +01:00
bool was_conversion;
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
String tmpstr(buff,sizeof(buff), &my_charset_bin);
http://bugs.mysql.com/bug.php?id=2368 Multibyte charsets do not check that incoming data is well-formed
2004-02-10 12:42:46 +01:00
uint copy_length;
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
uint32 not_used;
field.cc: http://bugs.mysql.com/bug.php?id=2218 updating utf-8 text field generate nonsense chars Fix for the above bug.
2004-01-15 12:58:06 +01:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
/* Convert character set if necessary */
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
if ((was_conversion= String::needs_conversion(length, cs, field_charset,
&not_used)))
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
{
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
uint conv_errors;
Simple optimization nsure that delete works not only on table->record[0] for federated tables
2005-06-29 11:44:40 +02:00
if (tmpstr.copy(from, length, cs, field_charset, &conv_errors))
{
/* Fatal OOM error */
bzero(ptr,Field_blob::pack_length());
return -1;
}
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
from= tmpstr.ptr();
length= tmpstr.length();
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
if (conv_errors)
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
error= 2;
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
}
http://bugs.mysql.com/bug.php?id=2368 Multibyte charsets do not check that incoming data is well-formed
2004-02-10 12:42:46 +01:00
copy_length= max_data_length();
Changed wellformedlen to well_formed_len Fixed that blobs >16M can be inserted/updated Fixed bug when doing CREATE TEMPORARY TABLE ... LIKE
2004-02-17 00:35:17 +01:00
/*
Precision Math implementation
2005-02-08 23:50:45 +01:00
copy_length is OK as last argument to well_formed_len as this is never
Changed wellformedlen to well_formed_len Fixed that blobs >16M can be inserted/updated Fixed bug when doing CREATE TEMPORARY TABLE ... LIKE
2004-02-17 00:35:17 +01:00
used to limit the length of the data. The cut of long data is done with
the 'min()' call below.
*/
copy_length= field_charset->cset->well_formed_len(field_charset,
Adding a new parameter for well_formed_length to return error. We'll use it for better warnign reporting.
2005-04-06 08:53:15 +02:00
from,from +
min(length, copy_length),
copy_length,
Minor clean-ups for the previous commit.
2005-04-06 12:12:44 +02:00
&well_formed_error);
http://bugs.mysql.com/bug.php?id=2368 Multibyte charsets do not check that incoming data is well-formed
2004-02-10 12:42:46 +01:00
if (copy_length < length)
func_str.result, func_str.test: Added a test case for bug #10124. sql_select.h, item_subselect.cc, sql_select.cc: Fixed bug #10124. The copy method of the store_key classes can return STORE_KEY_OK=0, STORE_KEY_FATAL=1, STORE_KEY_CONV=2 now. field.cc: Fixed bug #10124. When ussuing a warning the store methods return 2 instead of 1 now.
2005-06-23 15:15:50 +02:00
error= 2;
http://bugs.mysql.com/bug.php?id=2368 Multibyte charsets do not check that incoming data is well-formed
2004-02-10 12:42:46 +01:00
Field_blob::store_length(copy_length);
if (was_conversion || table->copy_blobs || copy_length <= MAX_FIELD_WIDTH)
Import changeset
2000-07-31 21:29:14 +02:00
{ // Must make a copy
Fixed some varnings from valgrind Set min value of max_allowed_packet to 1024 Fixed problem with UNION's without braces and SQL_CALC_FOUND_ROWS, LIMIT #,# and ORDER BY...LIMIT
2003-08-20 15:25:44 +02:00
if (from != value.ptr()) // For valgrind
{
http://bugs.mysql.com/bug.php?id=2368 Multibyte charsets do not check that incoming data is well-formed
2004-02-10 12:42:46 +01:00
value.copy(from,copy_length,charset());
merge with 4.0.15
2003-08-29 12:44:35 +02:00
from=value.ptr();
Fixed some varnings from valgrind Set min value of max_allowed_packet to 1024 Fixed problem with UNION's without braces and SQL_CALC_FOUND_ROWS, LIMIT #,# and ORDER BY...LIMIT
2003-08-20 15:25:44 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
}
bmove(ptr+packlength,(char*) &from,sizeof(char*));
}
Produce a "truncated" warning if a value cannot be converted into the column character set on INSERT/UPDATE.
2004-10-28 12:21:20 +02:00
if (error)
a fix (bug #9029 Traditional: Wrong SQLSTATE returned for string truncation).
2005-03-10 14:08:45 +01:00
{
if (table->in_use->abort_on_warning)
set_warning(MYSQL_ERROR::WARN_LEVEL_ERROR, ER_DATA_TOO_LONG, 1);
else
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
}
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_blob::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
CHARSET_INFO *cs=charset();
value.set(nr, 2, cs);
return Field_blob::store(value.ptr(),(uint) value.length(), cs);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_blob::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
CHARSET_INFO *cs=charset();
cleanup String::set(integer)
2006-06-14 23:09:03 +02:00
value.set(nr, unsigned_val, cs);
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
return Field_blob::store(value.ptr(), (uint) value.length(), cs);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
double Field_blob::val_real(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
char *end_not_used, *blob;
uint32 length;
CHARSET_INFO *cs;
Import changeset
2000-07-31 21:29:14 +02:00
memcpy_fixed(&blob,ptr+packlength,sizeof(char*));
if (!blob)
return 0.0;
Changed interface for my_strntod() to make it more general and more portable
2005-01-15 11:28:38 +01:00
length= get_length(ptr);
cs= charset();
After merge fixes Fix for BIT(X) field as string
2005-01-15 16:38:43 +01:00
return my_strntod(cs, blob, length, &end_not_used, &not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
longlong Field_blob::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
int not_used;
Import changeset
2000-07-31 21:29:14 +02:00
char *blob;
memcpy_fixed(&blob,ptr+packlength,sizeof(char*));
if (!blob)
return 0;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 length=get_length(ptr);
Changed my_strntoxxx functions to clear error number on start Allow one to change ANSI_QUOTES mode per thread and on the fly
2003-01-17 15:33:54 +01:00
return my_strntoll(charset(),blob,length,10,NULL,&not_used);
Import changeset
2000-07-31 21:29:14 +02:00
}
String *Field_blob::val_str(String *val_buffer __attribute__((unused)),
String *val_ptr)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
char *blob;
memcpy_fixed(&blob,ptr+packlength,sizeof(char*));
if (!blob)
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
val_ptr->set("",0,charset()); // A bit safer than ->length(0)
Import changeset
2000-07-31 21:29:14 +02:00
else
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
val_ptr->set((const char*) blob,get_length(ptr),charset());
Import changeset
2000-07-31 21:29:14 +02:00
return val_ptr;
}
cast.result, cast.test: Added test case for bug #11283. field.h, field.cc: Fixed bug #11283: wrong conversion from varchar to decimal. Added methods Field_string::val_decimal, Field_varstring::val_decimal, Field_blob::val_decimal. They are not inherited from the base class Field_longstr anymore.
2005-06-15 17:27:33 +02:00
my_decimal *Field_blob::val_decimal(my_decimal *decimal_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Fix warnings/compilation failures reported by Jan Lindstrom
2005-07-20 12:17:06 +02:00
const char *blob;
memcpy_fixed(&blob, ptr+packlength, sizeof(const char*));
cast.result, cast.test: Added test case for bug #11283. field.h, field.cc: Fixed bug #11283: wrong conversion from varchar to decimal. Added methods Field_string::val_decimal, Field_varstring::val_decimal, Field_blob::val_decimal. They are not inherited from the base class Field_longstr anymore.
2005-06-15 17:27:33 +02:00
if (!blob)
blob= "";
str2my_decimal(E_DEC_FATAL_ERROR, blob, get_length(ptr), charset(),
decimal_value);
return decimal_value;
}
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
int Field_blob::cmp(const char *a,uint32 a_length, const char *b,
uint32 b_length)
Import changeset
2000-07-31 21:29:14 +02:00
{
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*)a, a_length,
(const uchar*)b, b_length,
0);
Import changeset
2000-07-31 21:29:14 +02:00
}
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
int Field_blob::cmp_max(const char *a_ptr, const char *b_ptr,
uint max_length)
Import changeset
2000-07-31 21:29:14 +02:00
{
char *blob1,*blob2;
memcpy_fixed(&blob1,a_ptr+packlength,sizeof(char*));
memcpy_fixed(&blob2,b_ptr+packlength,sizeof(char*));
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
uint a_len= get_length(a_ptr), b_len= get_length(b_ptr);
set_if_smaller(a_len, max_length);
set_if_smaller(b_len, max_length);
return Field_blob::cmp(blob1,a_len,blob2,b_len);
Import changeset
2000-07-31 21:29:14 +02:00
}
int Field_blob::cmp_binary(const char *a_ptr, const char *b_ptr,
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 max_length)
Import changeset
2000-07-31 21:29:14 +02:00
{
char *a,*b;
uint diff;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 a_length,b_length;
Import changeset
2000-07-31 21:29:14 +02:00
memcpy_fixed(&a,a_ptr+packlength,sizeof(char*));
memcpy_fixed(&b,b_ptr+packlength,sizeof(char*));
a_length=get_length(a_ptr);
if (a_length > max_length)
a_length=max_length;
b_length=get_length(b_ptr);
if (b_length > max_length)
b_length=max_length;
diff=memcmp(a,b,min(a_length,b_length));
return diff ? diff : (int) (a_length - b_length);
}
/* The following is used only when comparing a key */
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
void Field_blob::get_key_image(char *buff, uint length, imagetype type)
Import changeset
2000-07-31 21:29:14 +02:00
{
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
uint32 blob_length= get_length(ptr);
Import changeset
2000-07-31 21:29:14 +02:00
char *blob;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
SCRUM: WL#1163 (Making spatial code optional) Pack of changes to do in sql/ code.
2004-01-15 18:06:22 +01:00
#ifdef HAVE_SPATIAL
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
if (type == itMBR)
Added GIS extension
2002-02-22 12:24:42 +01:00
{
Optimized GIS functions
2004-03-04 07:50:37 +01:00
const char *dummy;
Added GIS extension
2002-02-22 12:24:42 +01:00
MBR mbr;
Spatial code changed to get rid of inconsistent this->* operation Now we use virtual calls instead and redirect VMT pointer of the geometry object with 'new' operation
2004-03-12 09:04:00 +01:00
Geometry_buffer buffer;
Geometry *gobj;
Optimized GIS functions
2004-03-04 07:50:37 +01:00
if (blob_length < SRID_SIZE)
{
bzero(buff, SIZEOF_STORED_DOUBLE*4);
return;
}
get_ptr(&blob);
Additional fix for #12839
2005-10-31 09:05:27 +01:00
gobj= Geometry::construct(&buffer, blob, blob_length);
many warnings (practically safe but annoying) corrected
2006-01-03 17:54:54 +01:00
if (!gobj || gobj->get_mbr(&mbr, &dummy))
Optimized GIS functions
2004-03-04 07:50:37 +01:00
bzero(buff, SIZEOF_STORED_DOUBLE*4);
else
{
float8store(buff, mbr.xmin);
float8store(buff+8, mbr.xmax);
float8store(buff+16, mbr.ymin);
float8store(buff+24, mbr.ymax);
}
Added GIS extension
2002-02-22 12:24:42 +01:00
return;
}
SCRUM: WL#1163 (Making spatial code optional) Pack of changes to do in sql/ code.
2004-01-15 18:06:22 +01:00
#endif /*HAVE_SPATIAL*/
Added GIS extension
2002-02-22 12:24:42 +01:00
Bug #5832 SELECT doesn't return records in some cases
2004-10-05 18:22:14 +02:00
get_ptr(&blob);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
uint char_length= length / field_charset->mbmaxlen;
char_length= my_charpos(field_charset, blob, blob + blob_length,
char_length);
Bug #6043 erratic searching for diacriticals in indexed MyISAM UTF-8 table
2004-10-14 11:14:50 +02:00
set_if_smaller(blob_length, char_length);
Bug #5832 SELECT doesn't return records in some cases
2004-10-05 18:22:14 +02:00
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
if ((uint32) length > blob_length)
Fixes for mysqltest
2001-01-15 21:56:44 +01:00
{
Add CHARSET keyword (missing from last push)
2002-12-11 20:22:57 +01:00
/*
Must clear this as we do a memcmp in opt_range.cc to detect
identical keys
*/
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
bzero(buff+HA_KEY_BLOB_LENGTH+blob_length, (length-blob_length));
Import changeset
2000-07-31 21:29:14 +02:00
length=(uint) blob_length;
Fixes for mysqltest
2001-01-15 21:56:44 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
int2store(buff,length);
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
memcpy(buff+HA_KEY_BLOB_LENGTH, blob, length);
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
void Field_blob::set_key_image(char *buff,uint length)
Import changeset
2000-07-31 21:29:14 +02:00
{
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
length= uint2korr(buff);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
(void) Field_blob::store(buff+HA_KEY_BLOB_LENGTH, length, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
int Field_blob::key_cmp(const byte *key_ptr, uint max_key_length)
{
char *blob1;
uint blob_length=get_length(ptr);
memcpy_fixed(&blob1,ptr+packlength,sizeof(char*));
Bug #5832 SELECT doesn't return records in some cases
2004-10-05 18:22:14 +02:00
CHARSET_INFO *cs= charset();
uint char_length= max_key_length / cs->mbmaxlen;
char_length= my_charpos(cs, blob1, blob1+blob_length, char_length);
set_if_smaller(blob_length, char_length);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return Field_blob::cmp(blob1, blob_length,
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
(char*) key_ptr+HA_KEY_BLOB_LENGTH,
uint2korr(key_ptr));
Import changeset
2000-07-31 21:29:14 +02:00
}
int Field_blob::key_cmp(const byte *a,const byte *b)
{
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
return Field_blob::cmp((char*) a+HA_KEY_BLOB_LENGTH, uint2korr(a),
(char*) b+HA_KEY_BLOB_LENGTH, uint2korr(b));
Import changeset
2000-07-31 21:29:14 +02:00
}
Move handling of suffix_length from strnxfrm_bin() to filesort to ensure proper sorting of all kind of binary objects field::sort_key() now adds length last for varbinary/blob VARBINARY/BLOB is now sorted by filesort so that shorter strings comes before longer ones Fixed issues in test cases from last merge
2005-10-13 23:04:52 +02:00
uint32 Field_blob::sort_length() const
{
return (uint32) (current_thd->variables.max_sort_length +
(field_charset == &my_charset_bin ? 0 : packlength));
}
Import changeset
2000-07-31 21:29:14 +02:00
void Field_blob::sort_string(char *to,uint length)
{
char *blob;
uint blob_length=get_length();
field.cc: strnxfrm for binary and simple charsets
2003-01-16 12:52:37 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
if (!blob_length)
bzero(to,length);
else
{
Move handling of suffix_length from strnxfrm_bin() to filesort to ensure proper sorting of all kind of binary objects field::sort_key() now adds length last for varbinary/blob VARBINARY/BLOB is now sorted by filesort so that shorter strings comes before longer ones Fixed issues in test cases from last merge
2005-10-13 23:04:52 +02:00
if (field_charset == &my_charset_bin)
{
char *pos;
/*
Store length of blob last in blob to shorter blobs before longer blobs
*/
length-= packlength;
pos= to+length;
switch (packlength) {
case 1:
*pos= (char) blob_length;
break;
case 2:
mi_int2store(pos, blob_length);
break;
case 3:
mi_int3store(pos, blob_length);
break;
case 4:
mi_int4store(pos, blob_length);
break;
}
}
Import changeset
2000-07-31 21:29:14 +02:00
memcpy_fixed(&blob,ptr+packlength,sizeof(char*));
field.cc: strnxfrm for binary and simple charsets
2003-01-16 12:52:37 +01:00
blob_length=my_strnxfrm(field_charset,
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
(uchar*) to, length,
(uchar*) blob, blob_length);
bug#7284: strnxfrm returns different results for equal strings
2005-01-13 15:12:04 +01:00
DBUG_ASSERT(blob_length == length);
Import changeset
2000-07-31 21:29:14 +02:00
}
}
void Field_blob::sql_type(String &res) const
{
const char *str;
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
uint length;
Import changeset
2000-07-31 21:29:14 +02:00
switch (packlength) {
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
default: str="tiny"; length=4; break;
case 2: str=""; length=0; break;
case 3: str="medium"; length= 6; break;
case 4: str="long"; length=4; break;
}
MY_CS_NONTEXT flag for sprintf() incompatible charsets (ucs2 if the only one now) Some default_charset_info were removed
2003-06-03 11:59:17 +02:00
res.set_ascii(str,length);
Fix for SHOW, it didn't display CHARACTER SET and COLLATE clause for binary collations
2003-03-04 09:33:52 +01:00
if (charset() == &my_charset_bin)
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN("blob"));
Updates for multi-byte character sets (Note: test 'union' fails, but Sanja promised to fix this)
2003-01-14 13:28:36 +01:00
else
{
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN("text"));
Import changeset
2000-07-31 21:29:14 +02:00
}
}
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
char *Field_blob::pack(char *to, const char *from, uint max_length)
{
char *save=ptr;
ptr=(char*) from;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 length=get_length(); // Length of from string
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
if (length > max_length)
{
ptr=to;
length=max_length;
store_length(length); // Store max length
ptr=(char*) from;
}
else
memcpy(to,from,packlength); // Copy length
if (length)
{
get_ptr((char**) &from);
memcpy(to+packlength, from,length);
}
ptr=save; // Restore org row pointer
return to+packlength+length;
}
const char *Field_blob::unpack(char *to, const char *from)
{
memcpy(to,from,packlength);
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 length=get_length(from);
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
from+=packlength;
if (length)
memcpy_fixed(to+packlength, &from, sizeof(from));
else
bzero(to+packlength,sizeof(from));
return from+length;
}
Import changeset
2000-07-31 21:29:14 +02:00
/* Keys for blobs are like keys on varchars */
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int Field_blob::pack_cmp(const char *a, const char *b, uint key_length,
my_bool insert_or_update)
Import changeset
2000-07-31 21:29:14 +02:00
{
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
uint a_length, b_length;
Import changeset
2000-07-31 21:29:14 +02:00
if (key_length > 255)
{
a_length=uint2korr(a); a+=2;
b_length=uint2korr(b); b+=2;
}
else
{
a_length= (uint) (uchar) *a++;
b_length= (uint) (uchar) *b++;
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) a, a_length,
(const uchar*) b, b_length,
insert_or_update);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
int Field_blob::pack_cmp(const char *b, uint key_length,
my_bool insert_or_update)
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
{
char *a;
memcpy_fixed(&a,ptr+packlength,sizeof(char*));
if (!a)
return key_length > 0 ? -1 : 0;
uint a_length=get_length(ptr);
uint b_length;
if (key_length > 255)
{
b_length=uint2korr(b); b+=2;
}
else
b_length= (uint) (uchar) *b++;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return field_charset->coll->strnncollsp(field_charset,
(const uchar*) a, a_length,
(const uchar*) b, b_length,
insert_or_update);
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
}
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
/* Create a packed key that will be used for storage from a MySQL row */
Added bdb_lock_max and fixed bug in BDB tables when using key parts
2000-10-25 00:50:46 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
char *Field_blob::pack_key(char *to, const char *from, uint max_length)
{
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
char *save=ptr;
ptr=(char*) from;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uint32 length=get_length(); // Length of from string
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
uint char_length= ((field_charset->mbmaxlen > 1) ?
max_length/field_charset->mbmaxlen : max_length);
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
if (length)
get_ptr((char**) &from);
if (length > char_length)
char_length= my_charpos(field_charset, from, from+length, char_length);
set_if_smaller(length, char_length);
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
*to++= (uchar) length;
if (max_length > 255) // 2 byte length
*to++= (uchar) (length >> 8);
BDB: Bug#4531: unique key prefix interacts poorly with utf8, Bug#4594 column index make = failed for gbk
2004-08-24 15:32:57 +02:00
memcpy(to, from, length);
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
ptr=save; // Restore org row pointer
return to+length;
}
bug#2686 - index_merge select on BerkeleyDB table with varchar PK causes mysqld to crash. Added put_length() to get_length() and unpack_key() to pack_key(). Keys were packed with the minimum size of the length field for the key part and unpacked with length size of the base column. For the purpose of optimal key packing we have the method pack_key(), while rows are packed with pack(). Now keys are unpacked with unpack_key() and no longer with unpack() which is used for rows.
2004-06-24 14:54:28 +02:00
/*
Unpack a blob key into a record buffer.
SYNOPSIS
Field_blob::unpack_key()
to Pointer into the record buffer.
from Pointer to the packed key.
max_length Key length limit from key description.
DESCRIPTION
A blob key has a maximum size of 64K-1.
In its packed form, the length field is one or two bytes long,
depending on 'max_length'.
Depending on the maximum length of a blob, its length field is
put into 1 to 4 bytes. This is a property of the blob object,
described by 'packlength'.
Blobs are internally stored apart from the record buffer, which
contains a pointer to the blob buffer.
RETURN
Pointer into 'from' past the last byte copied from packed key.
*/
const char *Field_blob::unpack_key(char *to, const char *from, uint max_length)
{
/* get length of the blob key */
uint32 length= *((uchar*) from++);
if (max_length > 255)
length+= (*((uchar*) from++)) << 8;
/* put the length into the record buffer */
put_length(to, length);
/* put the address of the blob buffer or NULL */
if (length)
memcpy_fixed(to + packlength, &from, sizeof(from));
else
bzero(to + packlength, sizeof(from));
/* point to first byte of next field in 'from' */
return from + length;
}
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
/* Create a packed key that will be used for storage from a MySQL key */
char *Field_blob::pack_key_from_key_image(char *to, const char *from,
uint max_length)
{
uint length=uint2korr(from);
Import changeset
2000-07-31 21:29:14 +02:00
if (length > max_length)
length=max_length;
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*to++= (char) (length & 255);
Import changeset
2000-07-31 21:29:14 +02:00
if (max_length > 255)
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
*to++= (char) (length >> 8);
Import changeset
2000-07-31 21:29:14 +02:00
if (length)
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
memcpy(to, from+HA_KEY_BLOB_LENGTH, length);
Import changeset
2000-07-31 21:29:14 +02:00
return to+length;
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Portability fixes for Fortre C++ 5.0 (on Sun) in 32 and 64 bit modes.
2002-11-07 02:54:00 +01:00
uint Field_blob::packed_col_length(const char *data_ptr, uint length)
Fix that mysqladmin shutdown can be interrupted with ^C Fixed bug with BLOB keys in BDB tables Fixed problem with MERGE tables on OS with 32 bit files Fixed that TIME_TO_SEC() works with negative times
2001-11-03 14:18:09 +01:00
{
if (length > 255)
Portability fixes for Fortre C++ 5.0 (on Sun) in 32 and 64 bit modes.
2002-11-07 02:54:00 +01:00
return uint2korr(data_ptr)+2;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
return (uint) ((uchar) *data_ptr)+1;
Fix that mysqladmin shutdown can be interrupted with ^C Fixed bug with BLOB keys in BDB tables Fixed problem with MERGE tables on OS with 32 bit files Fixed that TIME_TO_SEC() works with negative times
2001-11-03 14:18:09 +01:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Fix that mysqladmin shutdown can be interrupted with ^C Fixed bug with BLOB keys in BDB tables Fixed problem with MERGE tables on OS with 32 bit files Fixed that TIME_TO_SEC() works with negative times
2001-11-03 14:18:09 +01:00
uint Field_blob::max_packed_col_length(uint max_length)
{
return (max_length > 255 ? 2 : 1)+max_length;
}
Fixed for bugs that was found when getting full code coverage of BDB Fixed bug with HEAP tables on windows Fixed bug with HAVING on empty tables
2001-01-17 02:15:20 +01:00
Optimized GIS functions
2004-03-04 07:50:37 +01:00
SCRUM: WL#1163 (Making spatial code optional) Pack of changes to do in sql/ code.
2004-01-15 18:06:22 +01:00
#ifdef HAVE_SPATIAL
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
void Field_geom::get_key_image(char *buff, uint length, imagetype type)
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
{
char *blob;
Optimized GIS functions
2004-03-04 07:50:37 +01:00
const char *dummy;
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
MBR mbr;
Cleanup/optimizations of structures and key usage to make it easier to move key-range-search to handler
2004-04-08 00:50:59 +02:00
ulong blob_length= get_length(ptr);
Geometry_buffer buffer;
Geometry *gobj;
Optimized GIS functions
2004-03-04 07:50:37 +01:00
if (blob_length < SRID_SIZE)
{
bzero(buff, SIZEOF_STORED_DOUBLE*4);
return;
}
get_ptr(&blob);
Fix for bug #12839 (Endian support is absurd)
2005-09-21 11:35:01 +02:00
gobj= Geometry::construct(&buffer, blob, blob_length);
many warnings (practically safe but annoying) corrected
2006-01-03 17:54:54 +01:00
if (!gobj || gobj->get_mbr(&mbr, &dummy))
Optimized GIS functions
2004-03-04 07:50:37 +01:00
bzero(buff, SIZEOF_STORED_DOUBLE*4);
else
{
float8store(buff, mbr.xmin);
float8store(buff + 8, mbr.xmax);
float8store(buff + 16, mbr.ymin);
float8store(buff + 24, mbr.ymax);
}
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
}
void Field_geom::sql_type(String &res) const
{
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
CHARSET_INFO *cs= &my_charset_latin1;
switch (geom_type)
{
case GEOM_POINT:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("point"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
break;
case GEOM_LINESTRING:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("linestring"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
break;
case GEOM_POLYGON:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("polygon"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
break;
case GEOM_MULTIPOINT:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("multipoint"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
break;
case GEOM_MULTILINESTRING:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("multilinestring"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
break;
case GEOM_MULTIPOLYGON:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("multipolygon"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
break;
case GEOM_GEOMETRYCOLLECTION:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("geometrycollection"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
break;
default:
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.set(STRING_WITH_LEN("geometry"), cs);
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
}
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
}
Fix for #6553 and #8776 (no errors on bad spatial data)
2005-04-29 17:23:02 +02:00
int Field_geom::store(double nr)
{
my_message(ER_CANT_CREATE_GEOMETRY_OBJECT,
ER(ER_CANT_CREATE_GEOMETRY_OBJECT), MYF(0));
return -1;
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_geom::store(longlong nr, bool unsigned_val)
Fix for #6553 and #8776 (no errors on bad spatial data)
2005-04-29 17:23:02 +02:00
{
my_message(ER_CANT_CREATE_GEOMETRY_OBJECT,
ER(ER_CANT_CREATE_GEOMETRY_OBJECT), MYF(0));
return -1;
}
int Field_geom::store_decimal(const my_decimal *)
{
my_message(ER_CANT_CREATE_GEOMETRY_OBJECT,
ER(ER_CANT_CREATE_GEOMETRY_OBJECT), MYF(0));
return -1;
}
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
int Field_geom::store(const char *from, uint length, CHARSET_INFO *cs)
{
if (!length)
bzero(ptr, Field_blob::pack_length());
else
{
Fix for #6553 and #8776 (no errors on bad spatial data)
2005-04-29 17:23:02 +02:00
if (from == Geometry::bad_geometry_data.ptr())
goto err;
Optimized GIS functions
2004-03-04 07:50:37 +01:00
// Check given WKB
uint32 wkb_type;
if (length < SRID_SIZE + WKB_HEADER_SIZE + SIZEOF_STORED_DOUBLE*2)
goto err;
Fix for bug #12839 (Endian support is absurd)
2005-09-21 11:35:01 +02:00
wkb_type= uint4korr(from + SRID_SIZE + 1);
Spatial code changed to get rid of inconsistent this->* operation Now we use virtual calls instead and redirect VMT pointer of the geometry object with 'new' operation
2004-03-12 09:04:00 +01:00
if (wkb_type < (uint32) Geometry::wkb_point ||
Optimized GIS functions
2004-03-04 07:50:37 +01:00
wkb_type > (uint32) Geometry::wkb_end)
Fix for #6553 and #8776 (no errors on bad spatial data)
2005-04-29 17:23:02 +02:00
goto err;
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
Field_blob::store_length(length);
if (table->copy_blobs || length <= MAX_FIELD_WIDTH)
{ // Must make a copy
value.copy(from, length, cs);
from= value.ptr();
}
bmove(ptr + packlength, (char*) &from, sizeof(char*));
}
return 0;
Optimized GIS functions
2004-03-04 07:50:37 +01:00
err:
bzero(ptr, Field_blob::pack_length());
Fix for #6553 and #8776 (no errors on bad spatial data)
2005-04-29 17:23:02 +02:00
my_message(ER_CANT_CREATE_GEOMETRY_OBJECT,
ER(ER_CANT_CREATE_GEOMETRY_OBJECT), MYF(0));
Fix for #233 (the second one)
2004-03-16 10:26:37 +01:00
return -1;
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
}
SCRUM: WL#1163 (Making spatial code optional) Pack of changes to do in sql/ code.
2004-01-15 18:06:22 +01:00
#endif /*HAVE_SPATIAL*/
SRID support. GeomertyFromWKB() function. SRID() function. ::store() methods for Field_geom. Code cleanup.
2003-03-18 12:30:32 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
/****************************************************************************
** enum type.
** This is a string which only can have a selection of different values.
** If one uses this string in a number context one gets the type number.
****************************************************************************/
enum ha_base_keytype Field_enum::key_type() const
{
switch (packlength) {
default: return HA_KEYTYPE_BINARY;
case 2: return HA_KEYTYPE_USHORT_INT;
case 3: return HA_KEYTYPE_UINT24;
case 4: return HA_KEYTYPE_ULONG_INT;
case 8: return HA_KEYTYPE_ULONGLONG;
}
}
void Field_enum::store_type(ulonglong value)
{
switch (packlength) {
case 1: ptr[0]= (uchar) value; break;
case 2:
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int2store(ptr,(unsigned short) value);
}
else
#endif
shortstore(ptr,(unsigned short) value);
break;
case 3: int3store(ptr,(long) value); break;
case 4:
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int4store(ptr,value);
}
else
#endif
longstore(ptr,(long) value);
break;
case 8:
#ifdef WORDS_BIGENDIAN
field.h, field.cc: Did some more split TABLE to TABLE and TABLE_SHARE
2005-01-11 00:17:21 +01:00
if (table->s->db_low_byte_first)
Import changeset
2000-07-31 21:29:14 +02:00
{
int8store(ptr,value);
}
else
#endif
longlongstore(ptr,value); break;
}
}
/*
** Note. Storing a empty string in a enum field gives a warning
** (if there isn't a empty value in the enum)
*/
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_enum::store(const char *from,uint length,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
int err= 0;
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
uint32 not_used;
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
String tmpstr(buff,sizeof(buff), &my_charset_bin);
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
/* Convert character set if necessary */
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
if (String::needs_conversion(length, cs, field_charset, &not_used))
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
{
A fix according to Monty's request: "uint *errors" is now a non-optional parameter in String:copy() and copy_and_convert().
2004-10-29 14:00:39 +02:00
uint dummy_errors;
tmpstr.copy(from, length, cs, field_charset, &dummy_errors);
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
from= tmpstr.ptr();
length= tmpstr.length();
}
Simplified 'wrong xxx name' error messages by introducing 'general' ER_WRONG_NAME error Cleaned up (and disabled part of) date/time/datetime format patch. One can't anymore change default read/write date/time/formats. This is becasue the non standard datetime formats can't be compared as strings and MySQL does still a lot of datetime comparisons as strings Changed flag argument to str_to_TIME() and get_date() from bool to uint Removed THD from str_to_xxxx functions and Item class. Fixed core dump when doing --print-defaults Move some common string functions to strfunc.cc Dates as strings are now of type my_charset_bin instead of default_charset() Introduce IDENT_QUOTED to not have to create an extra copy of simple identifiers (all chars < 128) Removed xxx_FORMAT_TYPE enums and replaced them with the old TIMESTAMP_xxx enums Renamed some TIMESTAMP_xxx enums to more appropriate names Use defines instead of integers for date/time/datetime string lengths Added to build system and use the new my_strtoll10() function.
2003-11-03 13:01:59 +01:00
/* Remove end space */
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
length= field_charset->cset->lengthsp(field_charset, from, length);
Bug#6202: ENUMs are not case sensitive even if declared BINARY
2004-10-25 14:51:26 +02:00
uint tmp=find_type2(typelib, from, length, field_charset);
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
if (!tmp)
Import changeset
2000-07-31 21:29:14 +02:00
{
refixed last vva patch with monty's instructions
2003-12-13 03:02:50 +01:00
if (length < 6) // Can't be more than 99999 enums
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
/* This is for reading numbers with LOAD DATA INFILE */
More use of new string->number conversion routines
2002-12-09 11:31:11 +01:00
char *end;
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
tmp=(uint) my_strntoul(cs,from,length,10,&end,&err);
if (err || end != from+length || tmp > typelib->count)
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
{
tmp=0;
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
}
else
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Import changeset
2000-07-31 21:29:14 +02:00
}
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
store_type((ulonglong) tmp);
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
return err;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_enum::store(double nr)
Import changeset
2000-07-31 21:29:14 +02:00
{
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
return Field_enum::store((longlong) nr, FALSE);
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_enum::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
if ((ulonglong) nr > typelib->count || nr == 0)
Import changeset
2000-07-31 21:29:14 +02:00
{
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Import changeset
2000-07-31 21:29:14 +02:00
nr=0;
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
error=1;
Import changeset
2000-07-31 21:29:14 +02:00
}
store_type((ulonglong) (uint) nr);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
double Field_enum::val_real(void)
{
return (double) Field_enum::val_int();
}
longlong Field_enum::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Import changeset
2000-07-31 21:29:14 +02:00
switch (packlength) {
case 1:
return (longlong) (uchar) ptr[0];
case 2:
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
{
uint16 tmp;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (table->s->db_low_byte_first)
tmp=sint2korr(ptr);
else
Import changeset
2000-07-31 21:29:14 +02:00
#endif
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
shortget(tmp,ptr);
return (longlong) tmp;
}
Import changeset
2000-07-31 21:29:14 +02:00
case 3:
return (longlong) uint3korr(ptr);
case 4:
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
{
uint32 tmp;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (table->s->db_low_byte_first)
tmp=uint4korr(ptr);
else
Import changeset
2000-07-31 21:29:14 +02:00
#endif
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
longget(tmp,ptr);
return (longlong) tmp;
}
Import changeset
2000-07-31 21:29:14 +02:00
case 8:
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
{
longlong tmp;
Import changeset
2000-07-31 21:29:14 +02:00
#ifdef WORDS_BIGENDIAN
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (table->s->db_low_byte_first)
tmp=sint8korr(ptr);
else
Import changeset
2000-07-31 21:29:14 +02:00
#endif
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
longlongget(tmp,ptr);
return tmp;
}
Import changeset
2000-07-31 21:29:14 +02:00
}
return 0; // impossible
}
String *Field_enum::val_str(String *val_buffer __attribute__((unused)),
String *val_ptr)
{
uint tmp=(uint) Field_enum::val_int();
if (!tmp || tmp > typelib->count)
Fixes after merge with 4.0 Cleaned up embedded library access and query cache handling Changed min stack size to 128K (to allow longer MyISAM keys) Fixed wrong priority for XOR (should be less than NEG to get -1^1 to work)
2003-12-19 15:25:50 +01:00
val_ptr->set("", 0, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
else
val_ptr->set((const char*) typelib->type_names[tmp-1],
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
typelib->type_lengths[tmp-1],
ENUM/SET fields now have charset too
2002-10-25 12:08:47 +02:00
field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
return val_ptr;
}
int Field_enum::cmp(const char *a_ptr, const char *b_ptr)
{
char *old=ptr;
ptr=(char*) a_ptr;
ulonglong a=Field_enum::val_int();
ptr=(char*) b_ptr;
ulonglong b=Field_enum::val_int();
ptr=old;
return (a < b) ? -1 : (a > b) ? 1 : 0;
}
void Field_enum::sort_string(char *to,uint length __attribute__((unused)))
{
ulonglong value=Field_enum::val_int();
to+=packlength-1;
for (uint i=0 ; i < packlength ; i++)
{
*to-- = (uchar) (value & 255);
value>>=8;
}
}
void Field_enum::sql_type(String &res) const
{
proper fix for the bug #2077: accented characters in enum/defaul values are reported incorrectly. note: bar asked me to use res.charset in ::sql_type() functions to be more consistent.
2003-12-16 14:39:33 +01:00
char buffer[255];
String enum_item(buffer, sizeof(buffer), res.charset());
Import changeset
2000-07-31 21:29:14 +02:00
res.length(0);
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN("enum("));
Import changeset
2000-07-31 21:29:14 +02:00
bool flag=0;
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
uint *len= typelib->type_lengths;
for (const char **pos= typelib->type_names; *pos; pos++, len++)
Import changeset
2000-07-31 21:29:14 +02:00
{
A fix according to Monty's request: "uint *errors" is now a non-optional parameter in String:copy() and copy_and_convert().
2004-10-29 14:00:39 +02:00
uint dummy_errors;
Import changeset
2000-07-31 21:29:14 +02:00
if (flag)
res.append(',');
proper fix for the bug #2077: accented characters in enum/defaul values are reported incorrectly. note: bar asked me to use res.charset in ::sql_type() functions to be more consistent.
2003-12-16 14:39:33 +01:00
/* convert to res.charset() == utf8, then quote */
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
enum_item.copy(*pos, *len, charset(), res.charset(), &dummy_errors);
proper fix for the bug #2077: accented characters in enum/defaul values are reported incorrectly. note: bar asked me to use res.charset in ::sql_type() functions to be more consistent.
2003-12-16 14:39:33 +01:00
append_unescaped(&res, enum_item.ptr(), enum_item.length());
flag= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
res.append(')');
}
Added support sql_mode, which can be used to produce various outputs of SHOW CREATE TABLE 'name'. Depending on the mode, the output can be compatible with various databases, including earlier versions of MySQL .
2003-01-16 01:04:50 +01:00
/*
set type.
This is a string which can have a collection of different values.
Each string value is separated with a ','.
For example "One,two,five"
If one uses this string in a number context one gets the bits as a longlong
number.
*/
Import changeset
2000-07-31 21:29:14 +02:00
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int Field_set::store(const char *from,uint length,CHARSET_INFO *cs)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.17
2003-12-17 16:35:34 +01:00
bool got_warning= 0;
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
int err= 0;
Added support sql_mode, which can be used to produce various outputs of SHOW CREATE TABLE 'name'. Depending on the mode, the output can be compatible with various databases, including earlier versions of MySQL .
2003-01-16 01:04:50 +01:00
char *not_used;
uint not_used2;
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
uint32 not_used_offset;
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
String tmpstr(buff,sizeof(buff), &my_charset_bin);
Import changeset
2000-07-31 21:29:14 +02:00
Precision Math implementation
2005-02-08 23:50:45 +01:00
/* Convert character set if necessary */
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
if (String::needs_conversion(length, cs, field_charset, &not_used_offset))
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
{
A fix according to Monty's request: "uint *errors" is now a non-optional parameter in String:copy() and copy_and_convert().
2004-10-29 14:00:39 +02:00
uint dummy_errors;
tmpstr.copy(from, length, cs, field_charset, &dummy_errors);
field.cc: Now all text fields convert data into proper charset before storing
2003-02-27 09:10:41 +01:00
from= tmpstr.ptr();
length= tmpstr.length();
}
Bug 6206: ENUMs are not case sensitive even if declared BINARY The same problem with SET columns: find_set() now executes find_type2() to do charset aware search, instead of always using system_charset_info comparison.
2004-10-26 10:17:37 +02:00
ulonglong tmp= find_set(typelib, from, length, field_charset,
&not_used, &not_used2, &got_warning);
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
if (!tmp && length && length < 22)
{
/* This is for reading numbers with LOAD DATA INFILE */
More use of new string->number conversion routines
2002-12-09 11:31:11 +01:00
char *end;
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
tmp=my_strntoull(cs,from,length,10,&end,&err);
if (err || end != from+length ||
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
tmp > (ulonglong) (((longlong) 1 << typelib->count) - (longlong) 1))
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
{
Support of warnings for all DML statements (Insert, Update and Alter) Fix LOAD DATA INFILE warnings to have a better meanigful messages Fix to make the mysql command line to automatically show the warnings count for all basic commands
2003-04-30 09:02:28 +02:00
tmp=0;
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
}
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
}
Merge with 4.0.17
2003-12-17 16:35:34 +01:00
else if (got_warning)
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Fixes for innobase usage Fixed bug when using TEXT columns with BDB tables Allow LOAD DATA INFILE to use numbers with ENUM and SET columns
2001-03-03 02:03:12 +01:00
store_type(tmp);
strnto family functions now return error in a new argument
2003-01-16 14:17:07 +01:00
return err;
Import changeset
2000-07-31 21:29:14 +02:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_set::store(longlong nr, bool unsigned_val)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Merge with 4.0.3 Some simple optimzations, more comments and indentation changes. Add ` around database in 'use database' in binary log. Moved max_error_count and max_warning_count to variables struct. Removed SHOW_WARNS_COUNT and SHOW_ERRORS_COUNT calls. Changed string functions to use character set of first string argument as default return characterset (Each string function can change the above assumption if needed)
2002-08-30 11:40:40 +02:00
int error= 0;
Import changeset
2000-07-31 21:29:14 +02:00
if ((ulonglong) nr > (ulonglong) (((longlong) 1 << typelib->count) -
(longlong) 1))
{
Support of warnings for all DML statements (Insert, Update and Alter) Fix LOAD DATA INFILE warnings to have a better meanigful messages Fix to make the mysql command line to automatically show the warnings count for all basic commands
2003-04-30 09:02:28 +02:00
nr&= (longlong) (((longlong) 1 << typelib->count) - (longlong) 1);
ER_WARN_DATA_TRUNCATED replaced with WARN_DATA_TRUNCATED.
2005-01-19 13:24:06 +01:00
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, WARN_DATA_TRUNCATED, 1);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
error=1;
Import changeset
2000-07-31 21:29:14 +02:00
}
store_type((ulonglong) nr);
Adding a necessary functionality to ::store and ::save_in_field that will take place properly after pull from 4.0, in order to handle conversions from quoted constants to bigint's.
2002-08-24 13:49:04 +02:00
return error;
Import changeset
2000-07-31 21:29:14 +02:00
}
String *Field_set::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
ulonglong tmp=(ulonglong) Field_enum::val_int();
uint bitnr=0;
val_buffer->length(0);
Fixed thar proper charset was not set in Field_set::val_str()
2004-12-06 13:22:51 +01:00
val_buffer->set_charset(field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
while (tmp && bitnr < (uint) typelib->count)
{
if (tmp & 1)
{
if (val_buffer->length())
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
val_buffer->append(&field_separator, 1, &my_charset_latin1);
Import changeset
2000-07-31 21:29:14 +02:00
String str(typelib->type_names[bitnr],
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
typelib->type_lengths[bitnr],
ENUM/SET fields now have charset too
2002-10-25 12:08:47 +02:00
field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
val_buffer->append(str);
}
tmp>>=1;
bitnr++;
}
return val_buffer;
}
void Field_set::sql_type(String &res) const
{
proper fix for the bug #2077: accented characters in enum/defaul values are reported incorrectly. note: bar asked me to use res.charset in ::sql_type() functions to be more consistent.
2003-12-16 14:39:33 +01:00
char buffer[255];
String set_item(buffer, sizeof(buffer), res.charset());
Import changeset
2000-07-31 21:29:14 +02:00
res.length(0);
Inefficient usage of String::append() fixed. Bad examples of usage of a string with its length fixed. The incorrect length in the trigger file configuration descriptor fixed (BUG#14090). A hook for unknown keys added to the parser to support old .TRG files.
2005-11-20 19:47:07 +01:00
res.append(STRING_WITH_LEN("set("));
Import changeset
2000-07-31 21:29:14 +02:00
bool flag=0;
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
uint *len= typelib->type_lengths;
for (const char **pos= typelib->type_names; *pos; pos++, len++)
Import changeset
2000-07-31 21:29:14 +02:00
{
A fix according to Monty's request: "uint *errors" is now a non-optional parameter in String:copy() and copy_and_convert().
2004-10-29 14:00:39 +02:00
uint dummy_errors;
Import changeset
2000-07-31 21:29:14 +02:00
if (flag)
res.append(',');
proper fix for the bug #2077: accented characters in enum/defaul values are reported incorrectly. note: bar asked me to use res.charset in ::sql_type() functions to be more consistent.
2003-12-16 14:39:33 +01:00
/* convert to res.charset() == utf8, then quote */
UCS2 support in ENUM and SET, which also fixes: Bug #5174 SHOW CREATE TABLE hangs up if the table contains half-with katakana enum values UCS2 values are stored in HEX encoding in FRM file
2004-12-06 17:45:32 +01:00
set_item.copy(*pos, *len, charset(), res.charset(), &dummy_errors);
proper fix for the bug #2077: accented characters in enum/defaul values are reported incorrectly. note: bar asked me to use res.charset in ::sql_type() functions to be more consistent.
2003-12-16 14:39:33 +01:00
append_unescaped(&res, set_item.ptr(), set_item.length());
flag= 1;
Import changeset
2000-07-31 21:29:14 +02:00
}
res.append(')');
}
/* returns 1 if the fields are equally defined */
bool Field::eq_def(Field *field)
{
filesort.cc, sql_analyse.cc, item_func.cc, item_strfunc.cc, field.cc: Unnesessary binary() calls have been removed
2003-03-13 12:34:46 +01:00
if (real_type() != field->real_type() || charset() != field->charset() ||
Import changeset
2000-07-31 21:29:14 +02:00
pack_length() != field->pack_length())
return 0;
return 1;
}
bool Field_enum::eq_def(Field *field)
{
if (!Field::eq_def(field))
return 0;
TYPELIB *from_lib=((Field_enum*) field)->typelib;
Update copyright Fixed memory leak on shutdown (Affects the embedded version & MyODBC)
2001-12-06 13:10:51 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
if (typelib->count < from_lib->count)
return 0;
for (uint i=0 ; i < from_lib->count ; i++)
Bug #4555 ALTER TABLE crashes mysqld with enum column collated utf8_unicode_ci
2004-07-23 16:10:06 +02:00
if (my_strnncoll(field_charset,
(const uchar*)typelib->type_names[i],
fixes for windows 64-bit compiler warnings
2005-06-13 12:41:15 +02:00
(uint) strlen(typelib->type_names[i]),
Bug #4555 ALTER TABLE crashes mysqld with enum column collated utf8_unicode_ci
2004-07-23 16:10:06 +02:00
(const uchar*)from_lib->type_names[i],
fixes for windows 64-bit compiler warnings
2005-06-13 12:41:15 +02:00
(uint) strlen(from_lib->type_names[i])))
Import changeset
2000-07-31 21:29:14 +02:00
return 0;
return 1;
}
bool Field_num::eq_def(Field *field)
Update copyright Fixed memory leak on shutdown (Affects the embedded version & MyODBC)
2001-12-06 13:10:51 +01:00
{
Import changeset
2000-07-31 21:29:14 +02:00
if (!Field::eq_def(field))
return 0;
Field_num *from_num= (Field_num*) field;
if (unsigned_flag != from_num->unsigned_flag ||
zerofill && !from_num->zerofill && !zero_pack() ||
dec != from_num->dec)
return 0;
return 1;
}
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
uint Field_num::is_equal(create_field *new_field)
{
Fix for bug#18133: Adding index to unsigned int causes on-line add index not to be used
2006-04-03 13:25:28 +02:00
return ((new_field->sql_type == real_type()) &&
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
((new_field->flags & UNSIGNED_FLAG) == (uint) (flags &
UNSIGNED_FLAG)) &&
((new_field->flags & AUTO_INCREMENT_FLAG) ==
(uint) (flags & AUTO_INCREMENT_FLAG)) &&
Fix for bug#18133: Adding index to unsigned int causes on-line add index not to be used
2006-04-03 13:25:28 +02:00
(new_field->length <= max_length()));
Faster alter table code for 5.1.
2005-07-22 22:43:59 +02:00
}
bit type
2004-12-17 15:06:05 +01:00
/*
Bit field.
We store the first 0 - 6 uneven bits among the null bits
at the start of the record. The rest bytes are stored in
the record itself.
For example:
CREATE TABLE t1 (a int, b bit(17), c bit(21) not null, d bit(8));
We would store data as follows in the record:
Byte Bit
1 7 - reserve for delete
6 - null bit for 'a'
5 - null bit for 'b'
4 - first (high) bit of 'b'
3 - first (high) bit of 'c'
2 - second bit of 'c'
1 - third bit of 'c'
0 - forth bit of 'c'
2 7 - firth bit of 'c'
6 - null bit for 'd'
3 - 6 four bytes for 'a'
7 - 8 two bytes for 'b'
9 - 10 two bytes for 'c'
11 one byte for 'd'
*/
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
Field_bit::Field_bit(char *ptr_arg, uint32 len_arg, uchar *null_ptr_arg,
uchar null_bit_arg, uchar *bit_ptr_arg, uchar bit_ofs_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
enum utype unireg_check_arg, const char *field_name_arg)
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
: Field(ptr_arg, len_arg, null_ptr_arg, null_bit_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check_arg, field_name_arg),
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
bit_ptr(bit_ptr_arg), bit_ofs(bit_ofs_arg), bit_len(len_arg & 7),
bytes_in_rec(len_arg / 8)
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
{
/*
Ensure that Field::eq() can distinguish between two different bit fields.
(two bit fields that are not null, may have same ptr and null_ptr)
*/
if (!null_ptr_arg)
null_bit= bit_ofs_arg;
}
Field *Field_bit::new_key_field(MEM_ROOT *root,
struct st_table *new_table,
char *new_ptr, uchar *new_null_ptr,
uint new_null_bit)
{
Field_bit *res;
if ((res= (Field_bit*) Field::new_key_field(root, new_table,
new_ptr, new_null_ptr,
new_null_bit)))
{
/* Move bits normally stored in null_pointer to new_ptr */
res->bit_ptr= (uchar*) new_ptr;
res->bit_ofs= 0;
if (bit_len)
res->ptr++; // Store rest of data here
}
return res;
}
bit type
2004-12-17 15:06:05 +01:00
int Field_bit::store(const char *from, uint length, CHARSET_INFO *cs)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
bit type
2004-12-17 15:06:05 +01:00
int delta;
Fix for BUG#14703 "Valgrind error when inserting 0 into a BIT column (like in type_bit.test)": test "length" first (otherwise when "length" is 0, the *from invalid access still triggers a Valgrind warning). I wrote to the Valgrind authors in case this is something fixable in Valgrind (normally the decision to issue a warning is based on the simulated CPU condition code, which should not be undefined here).
2005-11-07 16:18:46 +01:00
for (; length && !*from; from++, length--); // skip left 0's
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
delta= bytes_in_rec - length;
bit type
2004-12-17 15:06:05 +01:00
if (delta < -1 ||
(delta == -1 && (uchar) *from > ((1 << bit_len) - 1)) ||
(!bit_len && delta < 0))
{
set_rec_bits(0xff, bit_ptr, bit_ofs, bit_len);
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
memset(ptr, 0xff, bytes_in_rec);
a fix (bug #11546: Bad error message from inserting out of range values, SQL_MODE='tradition').
2005-07-26 09:18:35 +02:00
if (table->in_use->really_abort_on_warning())
set_warning(MYSQL_ERROR::WARN_LEVEL_ERROR, ER_DATA_TOO_LONG, 1);
else
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
bit type
2004-12-17 15:06:05 +01:00
return 1;
}
/* delta is >= -1 here */
if (delta > 0)
{
if (bit_len)
clr_rec_bits(bit_ptr, bit_ofs, bit_len);
bzero(ptr, delta);
memcpy(ptr + delta, from, length);
}
else if (delta == 0)
{
if (bit_len)
clr_rec_bits(bit_ptr, bit_ofs, bit_len);
memcpy(ptr, from, length);
}
else
{
if (bit_len)
{
set_rec_bits((uchar) *from, bit_ptr, bit_ofs, bit_len);
from++;
}
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
memcpy(ptr, from, bytes_in_rec);
bit type
2004-12-17 15:06:05 +01:00
}
return 0;
}
int Field_bit::store(double nr)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
return Field_bit::store((longlong) nr, FALSE);
bit type
2004-12-17 15:06:05 +01:00
}
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
int Field_bit::store(longlong nr, bool unsigned_val)
bit type
2004-12-17 15:06:05 +01:00
{
char buf[8];
mi_int8store(buf, nr);
return store(buf, 8, NULL);
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
int Field_bit::store_decimal(const my_decimal *val)
{
int err= 0;
longlong i= convert_decimal2longlong(val, 1, &err);
return test(err | store(i));
}
bit type
2004-12-17 15:06:05 +01:00
double Field_bit::val_real(void)
{
return (double) Field_bit::val_int();
}
longlong Field_bit::val_int(void)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
bit type
2004-12-17 15:06:05 +01:00
ulonglong bits= 0;
if (bit_len)
Give warnings if wrong date/time/datetime argument for STR_TO_DATE Small fixes while doing review of new pushed code More test cases for decimal
2005-03-30 15:00:31 +02:00
{
bit type
2004-12-17 15:06:05 +01:00
bits= get_rec_bits(bit_ptr, bit_ofs, bit_len);
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
bits<<= (bytes_in_rec * 8);
Give warnings if wrong date/time/datetime argument for STR_TO_DATE Small fixes while doing review of new pushed code More test cases for decimal
2005-03-30 15:00:31 +02:00
}
bit type
2004-12-17 15:06:05 +01:00
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
switch (bytes_in_rec) {
bit type
2004-12-17 15:06:05 +01:00
case 0: return bits;
case 1: return bits | (ulonglong) (uchar) ptr[0];
case 2: return bits | mi_uint2korr(ptr);
case 3: return bits | mi_uint3korr(ptr);
case 4: return bits | mi_uint4korr(ptr);
case 5: return bits | mi_uint5korr(ptr);
case 6: return bits | mi_uint6korr(ptr);
case 7: return bits | mi_uint7korr(ptr);
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
default: return mi_uint8korr(ptr + bytes_in_rec - sizeof(longlong));
bit type
2004-12-17 15:06:05 +01:00
}
}
String *Field_bit::val_str(String *val_buffer,
String *val_ptr __attribute__((unused)))
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
After merge fixes Fix for BIT(X) field as string
2005-01-15 16:38:43 +01:00
char buff[sizeof(longlong)];
bit type
2004-12-17 15:06:05 +01:00
uint length= min(pack_length(), sizeof(longlong));
ulonglong bits= val_int();
After merge fixes Fix for BIT(X) field as string
2005-01-15 16:38:43 +01:00
mi_int8store(buff,bits);
bit type
2004-12-17 15:06:05 +01:00
val_buffer->alloc(length);
After merge fixes Fix for BIT(X) field as string
2005-01-15 16:38:43 +01:00
memcpy_fixed((char*) val_buffer->ptr(), buff+8-length, length);
bit type
2004-12-17 15:06:05 +01:00
val_buffer->length(length);
val_buffer->set_charset(&my_charset_bin);
return val_buffer;
}
Precision Math implementation
2005-02-08 23:50:45 +01:00
my_decimal *Field_bit::val_decimal(my_decimal *deciaml_value)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_READ;
Precision Math implementation
2005-02-08 23:50:45 +01:00
int2my_decimal(E_DEC_FATAL_ERROR, val_int(), 1, deciaml_value);
return deciaml_value;
}
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
/*
Compare two bit fields using pointers within the record.
SYNOPSIS
cmp_max()
a Pointer to field->ptr in first record
b Pointer to field->ptr in second record
max_len Maximum length used in index
DESCRIPTION
This method is used from key_rec_cmp used by merge sorts used
by partitioned index read and later other similar places.
The a and b pointer must be pointers to the field in a record
(not the table->record[0] necessarily)
*/
int Field_bit::cmp_max(const char *a, const char *b, uint max_len)
{
my_ptrdiff_t a_diff= a - ptr;
my_ptrdiff_t b_diff= b - ptr;
if (bit_len)
{
int flag;
uchar bits_a= get_rec_bits(bit_ptr+a_diff, bit_ofs, bit_len);
uchar bits_b= get_rec_bits(bit_ptr+b_diff, bit_ofs, bit_len);
if ((flag= (int) (bits_a - bits_b)))
return flag;
}
return memcmp(a, b, field_length);
}
bit type
2004-12-17 15:06:05 +01:00
int Field_bit::key_cmp(const byte *str, uint length)
{
if (bit_len)
{
int flag;
uchar bits= get_rec_bits(bit_ptr, bit_ofs, bit_len);
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
if ((flag= (int) (bits - *(uchar*) str)))
bit type
2004-12-17 15:06:05 +01:00
return flag;
str++;
length--;
}
Bit type: code cleanup.
2005-04-12 16:36:21 +02:00
return memcmp(ptr, str, length);
bit type
2004-12-17 15:06:05 +01:00
}
int Field_bit::cmp_offset(uint row_offset)
{
if (bit_len)
{
int flag;
uchar bits_a= get_rec_bits(bit_ptr, bit_ofs, bit_len);
uchar bits_b= get_rec_bits(bit_ptr + row_offset, bit_ofs, bit_len);
if ((flag= (int) (bits_a - bits_b)))
return flag;
}
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
return memcmp(ptr, ptr + row_offset, bytes_in_rec);
bit type
2004-12-17 15:06:05 +01:00
}
void Field_bit::get_key_image(char *buff, uint length, imagetype type)
{
if (bit_len)
{
uchar bits= get_rec_bits(bit_ptr, bit_ofs, bit_len);
*buff++= bits;
length--;
}
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
memcpy(buff, ptr, min(length, bytes_in_rec));
bit type
2004-12-17 15:06:05 +01:00
}
void Field_bit::sql_type(String &res) const
{
CHARSET_INFO *cs= res.charset();
ulong length= cs->cset->snprintf(cs, (char*) res.ptr(), res.alloced_length(),
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
"bit(%d)", (int) field_length);
bit type
2004-12-17 15:06:05 +01:00
res.length((uint) length);
}
char *Field_bit::pack(char *to, const char *from, uint max_length)
{
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
DBUG_ASSERT(max_length);
uint length;
bit type
2004-12-17 15:06:05 +01:00
if (bit_len)
{
uchar bits= get_rec_bits(bit_ptr, bit_ofs, bit_len);
*to++= bits;
}
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
length= min(bytes_in_rec, max_length - (bit_len > 0));
memcpy(to, from, length);
bit type
2004-12-17 15:06:05 +01:00
return to + length;
}
const char *Field_bit::unpack(char *to, const char *from)
{
if (bit_len)
{
set_rec_bits(*from, bit_ptr, bit_ofs, bit_len);
from++;
}
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
memcpy(to, from, bytes_in_rec);
return from + bytes_in_rec;
bit type
2004-12-17 15:06:05 +01:00
}
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
/*
Bit field support for non-MyISAM tables.
*/
Field_bit_as_char::Field_bit_as_char(char *ptr_arg, uint32 len_arg,
many warnings (practically safe but annoying) corrected
2006-01-03 17:54:54 +01:00
uchar *null_ptr_arg, uchar null_bit_arg,
enum utype unireg_check_arg,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
const char *field_name_arg)
Merge mysql.com:/home/kostja/mysql/tmp_merge into mysql.com:/home/kostja/mysql/mysql-5.1-merge
2006-02-02 14:57:34 +01:00
:Field_bit(ptr_arg, len_arg, null_ptr_arg, null_bit_arg, 0, 0,
Remove extra ","
2006-04-11 10:03:04 +02:00
unireg_check_arg, field_name_arg)
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
{
bit_len= 0;
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
bytes_in_rec= (len_arg + 7) / 8;
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
}
int Field_bit_as_char::store(const char *from, uint length, CHARSET_INFO *cs)
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
ASSERT_COLUMN_MARKED_FOR_WRITE;
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
int delta;
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
uchar bits= field_length & 7;
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
Fix for BUG#14703 "Valgrind error when inserting 0 into a BIT column (like in type_bit.test)": test "length" first (otherwise when "length" is 0, the *from invalid access still triggers a Valgrind warning). I wrote to the Valgrind authors in case this is something fixable in Valgrind (normally the decision to issue a warning is based on the simulated CPU condition code, which should not be undefined here).
2005-11-07 16:18:46 +01:00
for (; length && !*from; from++, length--); // skip left 0's
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
delta= bytes_in_rec - length;
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
if (delta < 0 ||
(delta == 0 && bits && (uint) (uchar) *from >= (uint) (1 << bits)))
{
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
memset(ptr, 0xff, bytes_in_rec);
a fix (bug #10539: When inserting out of range value in BIT, different engines behaves differently).
2005-05-18 14:30:11 +02:00
if (bits)
*ptr&= ((1 << bits) - 1); /* set first byte */
a fix (bug #11546: Bad error message from inserting out of range values, SQL_MODE='tradition').
2005-07-26 09:18:35 +02:00
if (table->in_use->really_abort_on_warning())
set_warning(MYSQL_ERROR::WARN_LEVEL_ERROR, ER_DATA_TOO_LONG, 1);
else
set_warning(MYSQL_ERROR::WARN_LEVEL_WARN, ER_WARN_DATA_OUT_OF_RANGE, 1);
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
return 1;
}
bzero(ptr, delta);
memcpy(ptr + delta, from, length);
return 0;
}
void Field_bit_as_char::sql_type(String &res) const
{
CHARSET_INFO *cs= res.charset();
ulong length= cs->cset->snprintf(cs, (char*) res.ptr(), res.alloced_length(),
Bug #13601: Wrong int type for bit The wrong value was being reported as the field_length for BIT fields, resulting in confusion for at least Connector/J. The field_length is now always the number of bits in the field, as it should be.
2006-04-05 02:54:58 +02:00
"bit(%d)", (int) field_length);
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
res.length((uint) length);
}
Import changeset
2000-07-31 21:29:14 +02:00
/*****************************************************************************
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Handling of field and create_field
Import changeset
2000-07-31 21:29:14 +02:00
*****************************************************************************/
BUG#14139: When handling "CREATE TABLE(field_X type_spec,...) SELECT smth AS field_X, ...." avoid multiplying length of field_X by charset->mbmaxlen twice when calculating space required for field_X in the new table.
2005-10-25 22:56:17 +02:00
/*
Convert create_field::length from number of characters to number of bytes
SYNOPSIS
create_field::create_length_to_internal_length()
DESCRIPTION
A fix and a test case for Bug#13134 "Length of VARCHAR() utf8 column is increasing when table is recreated with PS/SP": make use of create_field::char_length more consistent in the code. Reinit create_field::length from create_field::char_length for every execution of a prepared statement (actually fixes the bug).
2006-02-21 17:52:20 +01:00
Convert create_field::length from number of characters to number of bytes.
BUG#14139: When handling "CREATE TABLE(field_X type_spec,...) SELECT smth AS field_X, ...." avoid multiplying length of field_X by charset->mbmaxlen twice when calculating space required for field_X in the new table.
2005-10-25 22:56:17 +02:00
*/
Fix: create table t1 (a char(10) character set cp1251) SELECT _koi8r'blabla' as a The above query created a field of koi8r charset, not cp1251 Change: CREATE TABLE a (a CHAR(1) CHARACTER SET utf8) Length now means character length, not byte length. The above creates a field that guarantees can store a multibyte value 1 character long. For utf8 the above creates a field that can store 3 bytes.
2003-08-11 15:18:34 +02:00
void create_field::create_length_to_internal_length(void)
{
Cleanups during review stage Added auto-correct of field length for enum/set tables for ALTER TABLE This is becasue of a bug in previous MySQL 4.1 versions where the length for enum/set was set incorrectly after ALTER TABLE
2004-12-02 13:43:51 +01:00
switch (sql_type) {
case MYSQL_TYPE_TINY_BLOB:
case MYSQL_TYPE_MEDIUM_BLOB:
case MYSQL_TYPE_LONG_BLOB:
case MYSQL_TYPE_BLOB:
case MYSQL_TYPE_VAR_STRING:
case MYSQL_TYPE_STRING:
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
case MYSQL_TYPE_VARCHAR:
Cleanups during review stage Added auto-correct of field length for enum/set tables for ALTER TABLE This is becasue of a bug in previous MySQL 4.1 versions where the length for enum/set was set incorrectly after ALTER TABLE
2004-12-02 13:43:51 +01:00
length*= charset->mbmaxlen;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
key_length= length;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
pack_length= calc_pack_length(sql_type, length);
Cleanups during review stage Added auto-correct of field length for enum/set tables for ALTER TABLE This is becasue of a bug in previous MySQL 4.1 versions where the length for enum/set was set incorrectly after ALTER TABLE
2004-12-02 13:43:51 +01:00
break;
case MYSQL_TYPE_ENUM:
case MYSQL_TYPE_SET:
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
/* Pack_length already calculated in sql_parse.cc */
Cleanups during review stage Added auto-correct of field length for enum/set tables for ALTER TABLE This is becasue of a bug in previous MySQL 4.1 versions where the length for enum/set was set incorrectly after ALTER TABLE
2004-12-02 13:43:51 +01:00
length*= charset->mbmaxlen;
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
key_length= pack_length;
Cleanups during review stage Added auto-correct of field length for enum/set tables for ALTER TABLE This is becasue of a bug in previous MySQL 4.1 versions where the length for enum/set was set incorrectly after ALTER TABLE
2004-12-02 13:43:51 +01:00
break;
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
case MYSQL_TYPE_BIT:
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
if (f_bit_as_char(pack_flag))
{
key_length= pack_length= ((length + 7) & ~7) / 8;
}
else
{
pack_length= length / 8;
/* We need one extra byte to store the bits we save among the null bits */
key_length= pack_length + test(length & 7);
}
Merge with global tree Fixed some found bugs in BIT fields Added more test cases for BIT fields and varchar
2004-12-19 19:25:19 +01:00
break;
Precision Math implementation
2005-02-08 23:50:45 +01:00
case MYSQL_TYPE_NEWDECIMAL:
Fixed bug 10465.
2005-05-12 16:37:38 +02:00
key_length= pack_length=
my_decimal_get_binary_size(my_decimal_length_to_precision(length,
decimals,
flags &
UNSIGNED_FLAG),
decimals);
Precision Math implementation
2005-02-08 23:50:45 +01:00
break;
Cleanups during review stage Added auto-correct of field length for enum/set tables for ALTER TABLE This is becasue of a bug in previous MySQL 4.1 versions where the length for enum/set was set incorrectly after ALTER TABLE
2004-12-02 13:43:51 +01:00
default:
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
key_length= pack_length= calc_pack_length(sql_type, length);
Cleanups during review stage Added auto-correct of field length for enum/set tables for ALTER TABLE This is becasue of a bug in previous MySQL 4.1 versions where the length for enum/set was set incorrectly after ALTER TABLE
2004-12-02 13:43:51 +01:00
break;
Fix: create table t1 (a char(10) character set cp1251) SELECT _koi8r'blabla' as a The above query created a field of koi8r charset, not cp1251 Change: CREATE TABLE a (a CHAR(1) CHARACTER SET utf8) Length now means character length, not byte length. The above creates a field that guarantees can store a multibyte value 1 character long. For utf8 the above creates a field that can store 3 bytes.
2003-08-11 15:18:34 +02:00
}
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
WL#926 "SUM(DISTINCT) and AVG(DISTINCT)": improvement of SUM(DISTINCT) and implementation of AVG(DISTINCT) which utilizes the approach with Fields. The patch implemented in October is portede to the up-to-date tree containing DECIMAL type. Tests for AVG(DISTINCT) (although there is not much to test provided that SUM(DISTINCT) works), cleanups for COUNT(DISTINCT) and GROUP_CONCAT() will follow in another changeset.
2005-03-13 21:50:43 +01:00
void create_field::init_for_tmp_table(enum_field_types sql_type_arg,
uint32 length_arg, uint32 decimals,
bool maybe_null, bool is_unsigned)
{
field_name= "";
sql_type= sql_type_arg;
After-merge fixes (Bug#13134)
2006-02-23 22:12:04 +01:00
char_length= length= length_arg;;
WL#926 "SUM(DISTINCT) and AVG(DISTINCT)": improvement of SUM(DISTINCT) and implementation of AVG(DISTINCT) which utilizes the approach with Fields. The patch implemented in October is portede to the up-to-date tree containing DECIMAL type. Tests for AVG(DISTINCT) (although there is not much to test provided that SUM(DISTINCT) works), cleanups for COUNT(DISTINCT) and GROUP_CONCAT() will follow in another changeset.
2005-03-13 21:50:43 +01:00
unireg_check= Field::NONE;
interval= 0;
charset= &my_charset_bin;
geom_type= Field::GEOM_GEOMETRY;
pack_flag= (FIELDFLAG_NUMBER |
((decimals & FIELDFLAG_MAX_DEC) << FIELDFLAG_DEC_SHIFT) |
(maybe_null ? FIELDFLAG_MAYBE_NULL : 0) |
(is_unsigned ? 0 : FIELDFLAG_DECIMAL));
}
Patch for WL#2894: Make stored routine variables work according to the standard. The idea is to use Field-classes to implement stored routines variables. Also, we should provide facade to Item-hierarchy by Item_field class (it is necessary, since SRVs take part in expressions). The patch fixes the following bugs: - BUG#8702: Stored Procedures: No Error/Warning shown for inappropriate data type matching; - BUG#8768: Functions: For any unsigned data type, -ve values can be passed and returned; - BUG#8769: Functions: For Int datatypes, out of range values can be passed and returned; - BUG#9078: STORED PROCDURE: Decimal digits are not displayed when we use DECIMAL datatype; - BUG#9572: Stored procedures: variable type declarations ignored; - BUG#12903: upper function does not work inside a function; - BUG#13705: parameters to stored procedures are not verified; - BUG#13808: ENUM type stored procedure parameter accepts non-enumerated data; - BUG#13909: Varchar Stored Procedure Parameter always BINARY string (ignores CHARACTER SET); - BUG#14161: Stored procedure cannot retrieve bigint unsigned; - BUG#14188: BINARY variables have no 0x00 padding; - BUG#15148: Stored procedure variables accept non-scalar values;
2005-12-07 15:01:17 +01:00
/*
Initialize field definition for create
SYNOPSIS
thd Thread handle
fld_name Field name
fld_type Field type
fld_length Field length
fld_decimals Decimal (if any)
fld_type_modifier Additional type information
fld_default_value Field default value (if any)
fld_on_update_value The value of ON UPDATE clause
fld_comment Field comment
fld_change Field change
fld_interval_list Interval list (if any)
fld_charset Field charset
fld_geom_type Field geometry type (if any)
RETURN
FALSE on success
TRUE on error
*/
bool create_field::init(THD *thd, char *fld_name, enum_field_types fld_type,
char *fld_length, char *fld_decimals,
uint fld_type_modifier, Item *fld_default_value,
Item *fld_on_update_value, LEX_STRING *fld_comment,
char *fld_change, List<String> *fld_interval_list,
CHARSET_INFO *fld_charset, uint fld_geom_type)
{
uint sign_len, allowed_type_modifier= 0;
ulong max_field_charlength= MAX_FIELD_CHARLENGTH;
DBUG_ENTER("create_field::init()");
field= 0;
field_name= fld_name;
def= fld_default_value;
flags= fld_type_modifier;
unireg_check= (fld_type_modifier & AUTO_INCREMENT_FLAG ?
Field::NEXT_NUMBER : Field::NONE);
decimals= fld_decimals ? (uint)atoi(fld_decimals) : 0;
if (decimals >= NOT_FIXED_DEC)
{
my_error(ER_TOO_BIG_SCALE, MYF(0), decimals, fld_name,
NOT_FIXED_DEC-1);
DBUG_RETURN(TRUE);
}
sql_type= fld_type;
length= 0;
change= fld_change;
interval= 0;
pack_length= key_length= 0;
charset= fld_charset;
geom_type= (Field::geometry_type) fld_geom_type;
interval_list.empty();
comment= *fld_comment;
/*
Set flag if this field doesn't have a default value
*/
if (!fld_default_value && !(fld_type_modifier & AUTO_INCREMENT_FLAG) &&
(fld_type_modifier & NOT_NULL_FLAG) && fld_type != FIELD_TYPE_TIMESTAMP)
flags|= NO_DEFAULT_VALUE_FLAG;
if (fld_length && !(length= (uint) atoi(fld_length)))
fld_length= 0; /* purecov: inspected */
sign_len= fld_type_modifier & UNSIGNED_FLAG ? 0 : 1;
switch (fld_type) {
case FIELD_TYPE_TINY:
if (!fld_length)
length= MAX_TINYINT_WIDTH+sign_len;
allowed_type_modifier= AUTO_INCREMENT_FLAG;
break;
case FIELD_TYPE_SHORT:
if (!fld_length)
length= MAX_SMALLINT_WIDTH+sign_len;
allowed_type_modifier= AUTO_INCREMENT_FLAG;
break;
case FIELD_TYPE_INT24:
if (!fld_length)
length= MAX_MEDIUMINT_WIDTH+sign_len;
allowed_type_modifier= AUTO_INCREMENT_FLAG;
break;
case FIELD_TYPE_LONG:
if (!fld_length)
length= MAX_INT_WIDTH+sign_len;
allowed_type_modifier= AUTO_INCREMENT_FLAG;
break;
case FIELD_TYPE_LONGLONG:
if (!fld_length)
length= MAX_BIGINT_WIDTH;
allowed_type_modifier= AUTO_INCREMENT_FLAG;
break;
case FIELD_TYPE_NULL:
break;
case FIELD_TYPE_NEWDECIMAL:
if (!fld_length && !decimals)
length= 10;
if (length > DECIMAL_MAX_PRECISION)
{
my_error(ER_TOO_BIG_PRECISION, MYF(0), length, fld_name,
DECIMAL_MAX_PRECISION);
DBUG_RETURN(TRUE);
}
if (length < decimals)
{
my_error(ER_M_BIGGER_THAN_D, MYF(0), fld_name);
DBUG_RETURN(TRUE);
}
length=
my_decimal_precision_to_length(length, decimals,
fld_type_modifier & UNSIGNED_FLAG);
pack_length=
my_decimal_get_binary_size(length, decimals);
break;
case MYSQL_TYPE_VARCHAR:
/*
Long VARCHAR's are automaticly converted to blobs in mysql_prepare_table
if they don't have a default value
*/
max_field_charlength= MAX_FIELD_VARCHARLENGTH;
break;
case MYSQL_TYPE_STRING:
break;
case FIELD_TYPE_BLOB:
case FIELD_TYPE_TINY_BLOB:
case FIELD_TYPE_LONG_BLOB:
case FIELD_TYPE_MEDIUM_BLOB:
case FIELD_TYPE_GEOMETRY:
if (fld_default_value)
{
/* Allow empty as default value. */
String str,*res;
res= fld_default_value->val_str(&str);
if (res->length())
{
my_error(ER_BLOB_CANT_HAVE_DEFAULT, MYF(0),
fld_name); /* purecov: inspected */
DBUG_RETURN(TRUE);
}
def= 0;
}
flags|= BLOB_FLAG;
break;
case FIELD_TYPE_YEAR:
if (!fld_length || length != 2)
length= 4; /* Default length */
flags|= ZEROFILL_FLAG | UNSIGNED_FLAG;
break;
case FIELD_TYPE_FLOAT:
/* change FLOAT(precision) to FLOAT or DOUBLE */
allowed_type_modifier= AUTO_INCREMENT_FLAG;
if (fld_length && !fld_decimals)
{
uint tmp_length= length;
if (tmp_length > PRECISION_FOR_DOUBLE)
{
my_error(ER_WRONG_FIELD_SPEC, MYF(0), fld_name);
DBUG_RETURN(TRUE);
}
else if (tmp_length > PRECISION_FOR_FLOAT)
{
sql_type= FIELD_TYPE_DOUBLE;
length= DBL_DIG+7; /* -[digits].E+### */
}
else
length= FLT_DIG+6; /* -[digits].E+## */
decimals= NOT_FIXED_DEC;
break;
}
if (!fld_length && !fld_decimals)
{
length= FLT_DIG+6;
decimals= NOT_FIXED_DEC;
}
if (length < decimals &&
decimals != NOT_FIXED_DEC)
{
my_error(ER_M_BIGGER_THAN_D, MYF(0), fld_name);
DBUG_RETURN(TRUE);
}
break;
case FIELD_TYPE_DOUBLE:
allowed_type_modifier= AUTO_INCREMENT_FLAG;
if (!fld_length && !fld_decimals)
{
length= DBL_DIG+7;
decimals= NOT_FIXED_DEC;
}
if (length < decimals &&
decimals != NOT_FIXED_DEC)
{
my_error(ER_M_BIGGER_THAN_D, MYF(0), fld_name);
DBUG_RETURN(TRUE);
}
break;
case FIELD_TYPE_TIMESTAMP:
if (!fld_length)
length= 14; /* Full date YYYYMMDDHHMMSS */
else if (length != 19)
{
/*
We support only even TIMESTAMP lengths less or equal than 14
and 19 as length of 4.1 compatible representation.
*/
length= ((length+1)/2)*2; /* purecov: inspected */
length= min(length,14); /* purecov: inspected */
}
flags|= ZEROFILL_FLAG | UNSIGNED_FLAG;
if (fld_default_value)
{
/* Grammar allows only NOW() value for ON UPDATE clause */
if (fld_default_value->type() == Item::FUNC_ITEM &&
((Item_func*)fld_default_value)->functype() == Item_func::NOW_FUNC)
{
unireg_check= (fld_on_update_value ? Field::TIMESTAMP_DNUN_FIELD:
Field::TIMESTAMP_DN_FIELD);
/*
We don't need default value any longer moreover it is dangerous.
Everything handled by unireg_check further.
*/
def= 0;
}
else
unireg_check= (fld_on_update_value ? Field::TIMESTAMP_UN_FIELD:
Field::NONE);
}
else
{
/*
If we have default TIMESTAMP NOT NULL column without explicit DEFAULT
or ON UPDATE values then for the sake of compatiblity we should treat
this column as having DEFAULT NOW() ON UPDATE NOW() (when we don't
have another TIMESTAMP column with auto-set option before this one)
or DEFAULT 0 (in other cases).
So here we are setting TIMESTAMP_OLD_FIELD only temporary, and will
replace this value by TIMESTAMP_DNUN_FIELD or NONE later when
information about all TIMESTAMP fields in table will be availiable.
If we have TIMESTAMP NULL column without explicit DEFAULT value
we treat it as having DEFAULT NULL attribute.
*/
unireg_check= (fld_on_update_value ? Field::TIMESTAMP_UN_FIELD :
(flags & NOT_NULL_FLAG ? Field::TIMESTAMP_OLD_FIELD :
Field::NONE));
}
break;
case FIELD_TYPE_DATE:
/* Old date type. */
if (protocol_version != PROTOCOL_VERSION-1)
sql_type= FIELD_TYPE_NEWDATE;
/* fall trough */
case FIELD_TYPE_NEWDATE:
length= 10;
break;
case FIELD_TYPE_TIME:
length= 10;
break;
case FIELD_TYPE_DATETIME:
length= 19;
break;
case FIELD_TYPE_SET:
{
if (fld_interval_list->elements > sizeof(longlong)*8)
{
my_error(ER_TOO_BIG_SET, MYF(0), fld_name); /* purecov: inspected */
DBUG_RETURN(TRUE);
}
pack_length= get_set_pack_length(fld_interval_list->elements);
List_iterator<String> it(*fld_interval_list);
String *tmp;
while ((tmp= it++))
interval_list.push_back(tmp);
/*
Set fake length to 1 to pass the below conditions.
Real length will be set in mysql_prepare_table()
when we know the character set of the column
*/
length= 1;
break;
}
case FIELD_TYPE_ENUM:
{
/* Should be safe. */
pack_length= get_enum_pack_length(fld_interval_list->elements);
List_iterator<String> it(*fld_interval_list);
String *tmp;
while ((tmp= it++))
interval_list.push_back(tmp);
length= 1; /* See comment for FIELD_TYPE_SET above. */
break;
}
case MYSQL_TYPE_VAR_STRING:
DBUG_ASSERT(0); /* Impossible. */
break;
case MYSQL_TYPE_BIT:
{
if (!fld_length)
length= 1;
if (length > MAX_BIT_FIELD_LENGTH)
{
my_error(ER_TOO_BIG_DISPLAYWIDTH, MYF(0), fld_name,
MAX_BIT_FIELD_LENGTH);
DBUG_RETURN(TRUE);
}
pack_length= (length + 7) / 8;
break;
}
case FIELD_TYPE_DECIMAL:
DBUG_ASSERT(0); /* Was obsolete */
}
After-merge fixes (Bug#13134)
2006-02-23 22:12:04 +01:00
/* Remember the value of length */
char_length= length;
Patch for WL#2894: Make stored routine variables work according to the standard. The idea is to use Field-classes to implement stored routines variables. Also, we should provide facade to Item-hierarchy by Item_field class (it is necessary, since SRVs take part in expressions). The patch fixes the following bugs: - BUG#8702: Stored Procedures: No Error/Warning shown for inappropriate data type matching; - BUG#8768: Functions: For any unsigned data type, -ve values can be passed and returned; - BUG#8769: Functions: For Int datatypes, out of range values can be passed and returned; - BUG#9078: STORED PROCDURE: Decimal digits are not displayed when we use DECIMAL datatype; - BUG#9572: Stored procedures: variable type declarations ignored; - BUG#12903: upper function does not work inside a function; - BUG#13705: parameters to stored procedures are not verified; - BUG#13808: ENUM type stored procedure parameter accepts non-enumerated data; - BUG#13909: Varchar Stored Procedure Parameter always BINARY string (ignores CHARACTER SET); - BUG#14161: Stored procedure cannot retrieve bigint unsigned; - BUG#14188: BINARY variables have no 0x00 padding; - BUG#15148: Stored procedure variables accept non-scalar values;
2005-12-07 15:01:17 +01:00
if (!(flags & BLOB_FLAG) &&
((length > max_field_charlength && fld_type != FIELD_TYPE_SET &&
fld_type != FIELD_TYPE_ENUM &&
(fld_type != MYSQL_TYPE_VARCHAR || fld_default_value)) ||
(!length &&
fld_type != MYSQL_TYPE_STRING &&
fld_type != MYSQL_TYPE_VARCHAR && fld_type != FIELD_TYPE_GEOMETRY)))
{
my_error((fld_type == MYSQL_TYPE_VAR_STRING ||
fld_type == MYSQL_TYPE_VARCHAR ||
fld_type == MYSQL_TYPE_STRING) ? ER_TOO_BIG_FIELDLENGTH :
ER_TOO_BIG_DISPLAYWIDTH,
MYF(0),
fld_name, max_field_charlength); /* purecov: inspected */
DBUG_RETURN(TRUE);
}
fld_type_modifier&= AUTO_INCREMENT_FLAG;
if ((~allowed_type_modifier) & fld_type_modifier)
{
my_error(ER_WRONG_FIELD_SPEC, MYF(0), fld_name);
DBUG_RETURN(TRUE);
}
DBUG_RETURN(FALSE); /* success */
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
enum_field_types get_blob_type_from_length(ulong length)
{
enum_field_types type;
if (length < 256)
type= FIELD_TYPE_TINY_BLOB;
else if (length < 65536)
type= FIELD_TYPE_BLOB;
else if (length < 256L*256L*256L)
type= FIELD_TYPE_MEDIUM_BLOB;
else
type= FIELD_TYPE_LONG_BLOB;
return type;
}
Import changeset
2000-07-31 21:29:14 +02:00
/*
removed init_count from IO_CACHE. Added missing mutex_unlock to slave replication code.
2002-06-05 22:04:38 +02:00
Make a field from the .frm file info
Import changeset
2000-07-31 21:29:14 +02:00
*/
uint32 calc_pack_length(enum_field_types type,uint32 length)
{
switch (type) {
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
case MYSQL_TYPE_VAR_STRING:
Update results for new varchar handling Fixed compiler warnings String results in CREATE ... SELECT are now created as CHAR(0), VARCHAR(X) or TEXT() depending on item->max_length
2004-12-07 14:47:00 +01:00
case MYSQL_TYPE_STRING:
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
case FIELD_TYPE_DECIMAL: return (length);
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
case MYSQL_TYPE_VARCHAR: return (length + (length < 256 ? 1: 2));
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_YEAR:
case FIELD_TYPE_TINY : return 1;
case FIELD_TYPE_SHORT : return 2;
case FIELD_TYPE_INT24:
case FIELD_TYPE_NEWDATE:
case FIELD_TYPE_TIME: return 3;
case FIELD_TYPE_TIMESTAMP:
case FIELD_TYPE_DATE:
case FIELD_TYPE_LONG : return 4;
case FIELD_TYPE_FLOAT : return sizeof(float);
case FIELD_TYPE_DOUBLE: return sizeof(double);
case FIELD_TYPE_DATETIME:
case FIELD_TYPE_LONGLONG: return 8; /* Don't crash if no longlong */
case FIELD_TYPE_NULL : return 0;
case FIELD_TYPE_TINY_BLOB: return 1+portable_sizeof_char_ptr;
field.cc: Geometry is now large blob
2003-04-03 20:19:15 +02:00
case FIELD_TYPE_BLOB: return 2+portable_sizeof_char_ptr;
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_MEDIUM_BLOB: return 3+portable_sizeof_char_ptr;
case FIELD_TYPE_LONG_BLOB: return 4+portable_sizeof_char_ptr;
field.cc: Geometry is now large blob
2003-04-03 20:19:15 +02:00
case FIELD_TYPE_GEOMETRY: return 4+portable_sizeof_char_ptr;
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_SET:
Precision Math implementation
2005-02-08 23:50:45 +01:00
case FIELD_TYPE_ENUM:
Remove compiler warnings and remove not used variables (Found during build process)
2005-02-25 15:53:22 +01:00
case FIELD_TYPE_NEWDECIMAL:
abort(); return 0; // This shouldn't happen
bit type
2004-12-17 15:06:05 +01:00
case FIELD_TYPE_BIT: return length / 8;
Remove compiler warnings and remove not used variables (Found during build process)
2005-02-25 15:53:22 +01:00
default:
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
}
uint pack_length_to_packflag(uint type)
{
switch (type) {
case 1: return f_settype((uint) FIELD_TYPE_TINY);
case 2: return f_settype((uint) FIELD_TYPE_SHORT);
case 3: return f_settype((uint) FIELD_TYPE_INT24);
case 4: return f_settype((uint) FIELD_TYPE_LONG);
case 8: return f_settype((uint) FIELD_TYPE_LONGLONG);
}
return 0; // This shouldn't happen
}
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
Field *make_field(TABLE_SHARE *share, char *ptr, uint32 field_length,
Update of query cache code. Changed some sql_alloc() -> thd->alloc() Removed a lot of compiler warnings on Linux Alpha (64 bit) Fixed some core dumps on 64 bit systems (wrong type for packet_len)
2001-12-05 12:03:00 +01:00
uchar *null_pos, uchar null_bit,
Import changeset
2000-07-31 21:29:14 +02:00
uint pack_flag,
Extension of .frm file (not yet ready for push)
2002-06-02 20:22:20 +02:00
enum_field_types field_type,
BINARY charset is now used instead of binary_flag
2002-10-25 10:58:32 +02:00
CHARSET_INFO *field_charset,
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
Field::geometry_type geom_type,
Import changeset
2000-07-31 21:29:14 +02:00
Field::utype unireg_check,
TYPELIB *interval,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
const char *field_name)
Import changeset
2000-07-31 21:29:14 +02:00
{
bit type
2004-12-17 15:06:05 +01:00
uchar *bit_ptr;
uchar bit_offset;
LINT_INIT(bit_ptr);
LINT_INIT(bit_offset);
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
if (field_type == FIELD_TYPE_BIT && !f_bit_as_char(pack_flag))
bit type
2004-12-17 15:06:05 +01:00
{
bit_ptr= null_pos;
bit_offset= null_bit;
if (f_maybe_null(pack_flag)) // if null field
{
bit_ptr+= (null_bit == 7); // shift bit_ptr and bit_offset
bit_offset= (bit_offset + 1) & 7;
}
}
Import changeset
2000-07-31 21:29:14 +02:00
if (!f_maybe_null(pack_flag))
{
null_pos=0;
null_bit=0;
}
bit type
2004-12-17 15:06:05 +01:00
else
{
null_bit= ((uchar) 1) << null_bit;
}
Fixed charsetnr sent to the client
2004-04-06 16:57:33 +02:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
switch (field_type) {
case FIELD_TYPE_DATE:
case FIELD_TYPE_NEWDATE:
case FIELD_TYPE_TIME:
case FIELD_TYPE_DATETIME:
case FIELD_TYPE_TIMESTAMP:
field_charset= &my_charset_bin;
default: break;
Fixed charsetnr sent to the client
2004-04-06 16:57:33 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
if (f_is_alpha(pack_flag))
{
if (!f_is_packed(pack_flag))
More debugging Print position in normal log for Binlog dump
2004-11-10 17:07:39 +01:00
{
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
if (field_type == MYSQL_TYPE_STRING ||
Fix to allow usage of 4.0 tables with 4.1
2004-11-12 18:01:49 +01:00
field_type == FIELD_TYPE_DECIMAL || // 3.23 or 4.0 string
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
field_type == MYSQL_TYPE_VAR_STRING)
More debugging Print position in normal log for Binlog dump
2004-11-10 17:07:39 +01:00
return new Field_string(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
More debugging Print position in normal log for Binlog dump
2004-11-10 17:07:39 +01:00
field_charset);
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
if (field_type == MYSQL_TYPE_VARCHAR)
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
return new Field_varstring(ptr,field_length,
HA_VARCHAR_PACKLENGTH(field_length),
null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
share,
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
field_charset);
More debugging Print position in normal log for Binlog dump
2004-11-10 17:07:39 +01:00
return 0; // Error
}
Import changeset
2000-07-31 21:29:14 +02:00
uint pack_length=calc_pack_length((enum_field_types)
f_packtype(pack_flag),
field_length);
SCRUM: WL#1163 (Making spatial code optional) Pack of changes to do in sql/ code.
2004-01-15 18:06:22 +01:00
#ifdef HAVE_SPATIAL
Added GIS extension
2002-02-22 12:24:42 +01:00
if (f_is_geom(pack_flag))
return new Field_geom(ptr,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name, share,
More OpenGIS compatibility: Point, MultiLine, Poligom, etc, are now not aliases for Geometry, but separate field types.
2003-03-27 10:09:09 +01:00
pack_length, geom_type);
SCRUM: WL#1163 (Making spatial code optional) Pack of changes to do in sql/ code.
2004-01-15 18:06:22 +01:00
#endif
Import changeset
2000-07-31 21:29:14 +02:00
if (f_is_blob(pack_flag))
return new Field_blob(ptr,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name, share,
Some fixes
2002-12-12 09:16:34 +01:00
pack_length, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
if (interval)
{
if (f_is_enum(pack_flag))
return new Field_enum(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
ENUM/SET fields now have charset too
2002-10-25 12:08:47 +02:00
pack_length, interval, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
else
return new Field_set(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
ENUM/SET fields now have charset too
2002-10-25 12:08:47 +02:00
pack_length, interval, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
}
}
Extension of .frm file (not yet ready for push)
2002-06-02 20:22:20 +02:00
switch (field_type) {
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_DECIMAL:
return new Field_decimal(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_decimals(pack_flag),
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag) == 0);
Precision Math implementation
2005-02-08 23:50:45 +01:00
case FIELD_TYPE_NEWDECIMAL:
return new Field_new_decimal(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Precision Math implementation
2005-02-08 23:50:45 +01:00
f_decimals(pack_flag),
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag) == 0);
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_FLOAT:
return new Field_float(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_decimals(pack_flag),
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag)== 0);
case FIELD_TYPE_DOUBLE:
return new Field_double(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_decimals(pack_flag),
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag)== 0);
case FIELD_TYPE_TINY:
return new Field_tiny(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag) == 0);
case FIELD_TYPE_SHORT:
return new Field_short(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag) == 0);
case FIELD_TYPE_INT24:
return new Field_medium(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag) == 0);
case FIELD_TYPE_LONG:
return new Field_long(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag) == 0);
case FIELD_TYPE_LONGLONG:
return new Field_longlong(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name,
Import changeset
2000-07-31 21:29:14 +02:00
f_is_zerofill(pack_flag) != 0,
f_is_dec(pack_flag) == 0);
case FIELD_TYPE_TIMESTAMP:
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
return new Field_timestamp(ptr,field_length, null_pos, null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name, share,
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_YEAR:
return new Field_year(ptr,field_length,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name);
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_DATE:
return new Field_date(ptr,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_NEWDATE:
return new Field_newdate(ptr,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_TIME:
return new Field_time(ptr,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_DATETIME:
return new Field_datetime(ptr,null_pos,null_bit,
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
unireg_check, field_name, field_charset);
Import changeset
2000-07-31 21:29:14 +02:00
case FIELD_TYPE_NULL:
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
return new Field_null(ptr, field_length, unireg_check, field_name,
field_charset);
bit type
2004-12-17 15:06:05 +01:00
case FIELD_TYPE_BIT:
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
return f_bit_as_char(pack_flag) ?
many warnings (practically safe but annoying) corrected
2006-01-03 17:54:54 +01:00
new Field_bit_as_char(ptr, field_length, null_pos, null_bit,
Merge mysql.com:/home/kostja/mysql/tmp_merge into mysql.com:/home/kostja/mysql/mysql-5.1-merge
2006-02-02 14:57:34 +01:00
unireg_check, field_name) :
Bit type support for non-MyISAM tables.
2005-04-12 09:27:43 +02:00
new Field_bit(ptr, field_length, null_pos, null_bit, bit_ptr,
Merge mysql.com:/home/kostja/mysql/tmp_merge into mysql.com:/home/kostja/mysql/mysql-5.1-merge
2006-02-02 14:57:34 +01:00
bit_offset, unireg_check, field_name);
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
Put temporary files in binlog cache when using BEGIN/COMMIT Let MySQL 4.0 read 4.1 .frm files without 4.1 specific extensions New variables @@rand_seed1 and @@rand_seed2 (used by replication) DROP TEMPORARY TABLE
2002-11-07 03:02:37 +01:00
default: // Impossible (Wrong version)
break;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge with 4.0
2002-11-21 14:56:48 +01:00
return 0;
Import changeset
2000-07-31 21:29:14 +02:00
}
/* Create a field suitable for create of table */
Fix of LEFT JOIN optimizer bug, problem with key cache on Windows and a lot of portability issues.
2000-10-14 02:16:35 +02:00
create_field::create_field(Field *old_field,Field *orig_field)
Import changeset
2000-07-31 21:29:14 +02:00
{
field= old_field;
field_name=change=old_field->field_name;
length= old_field->field_length;
flags= old_field->flags;
unireg_check=old_field->unireg_check;
pack_length=old_field->pack_length();
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
key_length= old_field->key_length();
Import changeset
2000-07-31 21:29:14 +02:00
sql_type= old_field->real_type();
Changes for new binary .frm format Fixes after last merge from 4.0. (Code not yet complete, need anoter merge from 4.0)
2002-06-04 07:23:57 +02:00
charset= old_field->charset(); // May be NULL ptr
comment= old_field->comment;
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
decimals= old_field->decimals();
Import changeset
2000-07-31 21:29:14 +02:00
/* Fix if the original table had 4 byte pointer blobs */
if (flags & BLOB_FLAG)
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
pack_length= (pack_length- old_field->table->s->blob_ptr_size +
Import changeset
2000-07-31 21:29:14 +02:00
portable_sizeof_char_ptr);
mysql_create_table now accepts field lengths in CREATE form, not in internal form. This is to simplify fixing indexes to mean number of chars rather than bytes too.
2003-09-10 14:25:26 +02:00
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
switch (sql_type) {
case FIELD_TYPE_BLOB:
switch (pack_length - portable_sizeof_char_ptr) {
case 1: sql_type= FIELD_TYPE_TINY_BLOB; break;
case 2: sql_type= FIELD_TYPE_BLOB; break;
case 3: sql_type= FIELD_TYPE_MEDIUM_BLOB; break;
default: sql_type= FIELD_TYPE_LONG_BLOB; break;
}
length=(length+charset->mbmaxlen-1) / charset->mbmaxlen;
key_length/= charset->mbmaxlen;
break;
Update results for new varchar handling Fixed compiler warnings String results in CREATE ... SELECT are now created as CHAR(0), VARCHAR(X) or TEXT() depending on item->max_length
2004-12-07 14:47:00 +01:00
case MYSQL_TYPE_STRING:
Changes for new binary .frm format Fixes after last merge from 4.0. (Code not yet complete, need anoter merge from 4.0)
2002-06-04 07:23:57 +02:00
/* Change CHAR -> VARCHAR if dynamic record length */
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
if (old_field->type() == MYSQL_TYPE_VAR_STRING)
sql_type= MYSQL_TYPE_VARCHAR;
/* fall through */
case MYSQL_TYPE_ENUM:
case MYSQL_TYPE_SET:
case MYSQL_TYPE_VARCHAR:
case MYSQL_TYPE_VAR_STRING:
Add 0x before pointers (to help with debugging) Add support for VARCHAR with 1 or 2 length bytes Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly) Give error if we got problems in temporary tables during a SELECT Don't use new table generated by ALTER TABLE if index generation fails Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
2004-12-18 04:19:21 +01:00
/* This is corrected in create_length_to_internal_length */
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
length= (length+charset->mbmaxlen-1) / charset->mbmaxlen;
break;
#ifdef HAVE_SPATIAL
case FIELD_TYPE_GEOMETRY:
geom_type= ((Field_geom*)old_field)->geom_type;
break;
#endif
default:
break;
Import changeset
2000-07-31 21:29:14 +02:00
}
Add support for up to VARCHAR (size up to 65535) Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors Added support for VARCHAR KEYS to heap Removed support for ISAM Now only long VARCHAR columns are changed to TEXT on demand (not CHAR) Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
2004-12-06 01:00:37 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
if (flags & (ENUM_FLAG | SET_FLAG))
interval= ((Field_enum*) old_field)->typelib;
else
interval=0;
Automatic repair of MyISAM tables + small bug fixes
2000-10-16 23:47:15 +02:00
def=0;
Merge mysql.com:/opt/local/work/mysql-4.1-13134 into mysql.com:/opt/local/work/mysql-5.0-runtime
2006-02-23 22:00:15 +01:00
char_length= length;
Added option --valgrind-mysqltest to mysql-test-run Added flag to Field::store(longlong) to specify if value is unsigned. This fixes bug #12750: Incorrect storage of 9999999999999999999 in DECIMAL(19, 0) Fixed warning from valgrind in CREATE ... SELECT Fixed double free of mysql.options if reconnect failed
2005-09-14 00:41:44 +02:00
NO_DEFAULT_VALUE_FLAG and BLOB_FLAG combined
2005-07-15 13:25:31 +02:00
if (!(flags & (NO_DEFAULT_VALUE_FLAG | BLOB_FLAG)) &&
fix (bug #11964: ALTER TABLE gives wrong error message with sql-mode TRADITIONAL (timestamp)).
2005-07-25 11:36:07 +02:00
old_field->ptr && orig_field &&
(sql_type != FIELD_TYPE_TIMESTAMP || /* set def only if */
old_field->table->timestamp_field != old_field || /* timestamp field */
unireg_check == Field::TIMESTAMP_UN_FIELD)) /* has default val */
Import changeset
2000-07-31 21:29:14 +02:00
{
char buff[MAX_FIELD_WIDTH],*pos;
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
String tmp(buff,sizeof(buff), charset), *res;
my_ptrdiff_t diff;
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
bool is_null;
Fix of LEFT JOIN optimizer bug, problem with key cache on Windows and a lot of portability issues.
2000-10-14 02:16:35 +02:00
INSERT ... UPDATE ... VALUES()
2003-05-03 01:16:56 +02:00
/* Get the value from default_values */
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
diff= (my_ptrdiff_t) (orig_field->table->s->default_values-
orig_field->table->record[0]);
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
orig_field->move_field_offset(diff); // Points now at default_values
Remove dflt_field from field structure as this was only needed when createing temporary table and I found another soultion that doesn't increase the size of the field structure for all table instances. (Better fix for bug #19089) Fixed compiler warnings Fixed valgrind warning in Item_date_add_intervall::eq. (Recoding of bugfix #19490)
2006-05-24 10:56:59 +02:00
if (!orig_field->is_real_null())
Fix of LEFT JOIN optimizer bug, problem with key cache on Windows and a lot of portability issues.
2000-10-14 02:16:35 +02:00
{
Merge a193-229-222-105.elisa-laajakaista.fi:/home/my/bk/mysql-5.0 into a193-229-222-105.elisa-laajakaista.fi:/home/my/bk/mysql-5.1-new
2006-05-30 15:07:49 +02:00
char buff[MAX_FIELD_WIDTH], *pos;
String tmp(buff, sizeof(buff), charset), *res;
Remove dflt_field from field structure as this was only needed when createing temporary table and I found another soultion that doesn't increase the size of the field structure for all table instances. (Better fix for bug #19089) Fixed compiler warnings Fixed valgrind warning in Item_date_add_intervall::eq. (Recoding of bugfix #19490)
2006-05-24 10:56:59 +02:00
res= orig_field->val_str(&tmp);
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
pos= (char*) sql_strmake(res->ptr(), res->length());
def= new Item_string(pos, res->length(), charset);
Fix of LEFT JOIN optimizer bug, problem with key cache on Windows and a lot of portability issues.
2000-10-14 02:16:35 +02:00
}
Merge a193-229-222-105.elisa-laajakaista.fi:/home/my/bk/mysql-5.0 into a193-229-222-105.elisa-laajakaista.fi:/home/my/bk/mysql-5.1-new
2006-05-30 15:07:49 +02:00
orig_field->move_field_offset(-diff); // Back to record[0]
Import changeset
2000-07-31 21:29:14 +02:00
}
}
Support of warnings for all DML statements (Insert, Update and Alter) Fix LOAD DATA INFILE warnings to have a better meanigful messages Fix to make the mysql command line to automatically show the warnings count for all basic commands
2003-04-30 09:02:28 +02:00
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
/*
maximum possible display length for blob
SYNOPSIS
Field_blob::max_length()
RETURN
length
*/
uint32 Field_blob::max_length()
{
switch (packlength)
{
case 1:
Merge mysql.com:/usr/home/bar/mysql-4.1.b15581 into mysql.com:/usr/home/bar/mysql-5.0
2006-01-12 08:38:09 +01:00
return 255 * field_charset->mbmaxlen;
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
case 2:
Merge mysql.com:/usr/home/bar/mysql-4.1.b15581 into mysql.com:/usr/home/bar/mysql-5.0
2006-01-12 08:38:09 +01:00
return 65535 * field_charset->mbmaxlen;
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
case 3:
Merge mysql.com:/usr/home/bar/mysql-4.1.b15581 into mysql.com:/usr/home/bar/mysql-5.0
2006-01-12 08:38:09 +01:00
return 16777215 * field_charset->mbmaxlen;
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
case 4:
return (uint32) 4294967295U;
default:
DBUG_ASSERT(0); // we should never go here
return 0;
}
}
/*****************************************************************************
Warning handling
*****************************************************************************/
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
/*
Produce warning or note about data saved into field
Precision Math implementation
2005-02-08 23:50:45 +01:00
SYNOPSIS
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning()
level - level of message (Note/Warning/Error)
code - error code of message to be produced
cuted_increment - whenever we should increase cut fields count or not
NOTE
This function won't produce warning and increase cut fields counter
if count_cuted_fields == FIELD_CHECK_IGNORE for current thread.
RETURN VALUE
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
1 if count_cuted_fields == FIELD_CHECK_IGNORE
0 otherwise
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
*/
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
bool
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Field::set_warning(MYSQL_ERROR::enum_warning_level level, uint code,
int cuted_increment)
Support of warnings for all DML statements (Insert, Update and Alter) Fix LOAD DATA INFILE warnings to have a better meanigful messages Fix to make the mysql command line to automatically show the warnings count for all basic commands
2003-04-30 09:02:28 +02:00
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
THD *thd= table->in_use;
After merge fixes Use server character set if --default-character-set is not used Added convert_string() for more efficient alloc+character-set convert of strings
2003-08-18 23:08:08 +02:00
if (thd->count_cuted_fields)
{
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
thd->cuted_fields+= cuted_increment;
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
push_warning_printf(thd, level, code, ER(code), field_name,
thd->row_count);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
return 0;
}
return 1;
}
/*
Produce warning or note about datetime string data saved into field
Precision Math implementation
2005-02-08 23:50:45 +01:00
SYNOPSIS
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
set_datime_warning()
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
level - level of message (Note/Warning/Error)
code - error code of message to be produced
str - string value which we tried to save
str_len - length of string which we tried to save
ts_type - type of datetime value (datetime/date/time)
cuted_increment - whenever we should increase cut fields count or not
NOTE
This function will always produce some warning but won't increase cut
fields counter if count_cuted_fields == FIELD_CHECK_IGNORE for current
thread.
*/
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
void
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Field::set_datetime_warning(MYSQL_ERROR::enum_warning_level level, uint code,
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
const char *str, uint str_length,
timestamp_type ts_type, int cuted_increment)
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (table->in_use->really_abort_on_warning() ||
set_warning(level, code, cuted_increment))
make_truncated_value_warning(table->in_use, str, str_length, ts_type,
field_name);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
}
/*
Produce warning or note about integer datetime value saved into field
Precision Math implementation
2005-02-08 23:50:45 +01:00
SYNOPSIS
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning()
level - level of message (Note/Warning/Error)
code - error code of message to be produced
nr - numeric value which we tried to save
ts_type - type of datetime value (datetime/date/time)
cuted_increment - whenever we should increase cut fields count or not
NOTE
This function will always produce some warning but won't increase cut
fields counter if count_cuted_fields == FIELD_CHECK_IGNORE for current
thread.
*/
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
void
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Field::set_datetime_warning(MYSQL_ERROR::enum_warning_level level, uint code,
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
longlong nr, timestamp_type ts_type,
int cuted_increment)
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (table->in_use->really_abort_on_warning() ||
set_warning(level, code, cuted_increment))
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
{
char str_nr[22];
char *str_end= longlong10_to_str(nr, str_nr, -10);
fixes for windows 64-bit compiler warnings
2005-06-13 12:41:15 +02:00
make_truncated_value_warning(table->in_use, str_nr,
(uint) (str_end - str_nr),
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
ts_type, field_name);
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
}
}
/*
Produce warning or note about double datetime data saved into field
Precision Math implementation
2005-02-08 23:50:45 +01:00
SYNOPSIS
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
set_warning()
level - level of message (Note/Warning/Error)
code - error code of message to be produced
nr - double value which we tried to save
ts_type - type of datetime value (datetime/date/time)
NOTE
This function will always produce some warning but won't increase cut
fields counter if count_cuted_fields == FIELD_CHECK_IGNORE for current
thread.
*/
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
void
Invalid DEFAULT values for CREATE TABLE now generates errors. (Bug #5902) CAST() now produces warnings when casting a wrong INTEGER or CHAR values. This also applies to implicite string to number casts. (Bug #5912) ALTER TABLE now fails in STRICT mode if it generates warnings. Inserting a zero date in a DATE, DATETIME or TIMESTAMP column during TRADITIONAL mode now produces an error. (Bug #5933)
2005-04-01 14:04:50 +02:00
Field::set_datetime_warning(MYSQL_ERROR::enum_warning_level level, uint code,
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
double nr, timestamp_type ts_type)
{
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
if (table->in_use->really_abort_on_warning() ||
set_warning(level, code, 1))
WL#1264 "Per-thread time zone support infrastructure". Added basic per-thread time zone functionality (based on public domain elsie-code). Now user can select current time zone (from the list of time zones described in system tables). All NOW-like functions honor this time zone, values of TIMESTAMP type are interpreted as values in this time zone, so now our TIMESTAMP type behaves similar to Oracle's TIMESTAMP WITH LOCAL TIME ZONE (or proper PostgresSQL type). WL#1266 "CONVERT_TZ() - basic time with time zone conversion function". Fixed problems described in Bug #2336 (Different number of warnings when inserting bad datetime as string or as number). This required reworking of datetime realted warning hadling (they now generated at Field object level not in conversion functions). Optimization: Now Field class descendants use table->in_use member instead of current_thd macro.
2004-06-18 08:11:31 +02:00
{
/* DBL_DIG is enough to print '-[digits].E+###' */
char str_nr[DBL_DIG + 8];
uint str_len= my_sprintf(str_nr, (str_nr, "%g", nr));
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
make_truncated_value_warning(table->in_use, str_nr, str_len, ts_type,
field_name);
After merge fixes Use server character set if --default-character-set is not used Added convert_string() for more efficient alloc+character-set convert of strings
2003-08-18 23:08:08 +02:00
}
Support of warnings for all DML statements (Insert, Update and Alter) Fix LOAD DATA INFILE warnings to have a better meanigful messages Fix to make the mysql command line to automatically show the warnings count for all basic commands
2003-04-30 09:02:28 +02:00
}
Reference in a new issue Copy permalink