mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-17 20:42:30 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/sql/sql_update.cc

1698 lines
50 KiB
C++
Raw Normal View History

my_strtoll10-x86.s: Corrected spelling in copyright text Makefile.am: Don't update the files from BitKeeper Many files: Removed "MySQL Finland AB & TCX DataKonsult AB" from copyright header Adjusted year(s) in copyright header Many files: Added GPL copyright text Removed files: Docs/Support/colspec-fix.pl Docs/Support/docbook-fixup.pl Docs/Support/docbook-prefix.pl Docs/Support/docbook-split Docs/Support/make-docbook Docs/Support/make-makefile Docs/Support/test-make-manual Docs/Support/test-make-manual-de Docs/Support/xwf
2006-12-31 01:02:27 +01:00
/* Copyright (C) 2000-2006 MySQL AB
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Many files: Changed header to GPL version 2 only
2006-12-23 20:17:15 +01:00
the Free Software Foundation; version 2 of the License.
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
Ensure that privileges are tested properly for multi-table-updates. Now one need only SELECT privilege for tables that are only read in UPDATE statements with many tables. (Bug #2377).
2004-02-03 19:17:23 +01:00
/*
Single table and multi table updates of tables.
Multi-table updates were introduced by Sinisa & Monty
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
*/
Import changeset
2000-07-31 21:29:14 +02:00
#include "mysql_priv.h"
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
#include "sql_select.h"
WL#1218 "Triggers". Some very preliminary version of patch. Mostly needed for Monty for him getting notion what needed for triggers from new .FRM format. Things to be done: - Right placement of trigger's invocations - Right handling of errors in triggers (including transaction rollback) - Support for priviliges - Right handling of DROP/RENAME table (hope that it will be handled automatically with merging of .TRG into .FRM file) - Saving/restoring some information critical for trigger creation and replication with their definitions (e.g. sql_mode, creator, ...) - Replication Already has some known bugs so probably not for general review.
2004-09-07 14:29:46 +02:00
#include "sp_head.h"
#include "sql_trigger.h"
Import changeset
2000-07-31 21:29:14 +02:00
/* Return 0 if row hasn't changed */
Bug#19978: INSERT .. ON DUPLICATE erroneously reports some records were updated. INSERT ... ON DUPLICATE KEY UPDATE reports that a record was updated when the duplicate key occurs even if the record wasn't actually changed because the update values are the same as those in the record. Now the compare_record() function is used to check whether the record was changed and the update of a record reported only if the record differs from the original one.
2007-02-06 22:46:03 +01:00
bool compare_record(TABLE *table, query_id_t query_id)
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixed memory reference errors found by valgrind
2005-01-12 02:38:53 +01:00
if (table->s->blob_fields + table->s->varchar_fields == 0)
INSERT ... UPDATE ... VALUES()
2003-05-03 01:16:56 +02:00
return cmp_record(table,record[1]);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/* Compare null bits */
Import changeset
2000-07-31 21:29:14 +02:00
if (memcmp(table->null_flags,
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
table->null_flags+table->s->rec_buff_length,
table->s->null_bytes))
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
return TRUE; // Diff in NULL value
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/* Compare updated fields */
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
for (Field **ptr= table->field ; *ptr ; ptr++)
Import changeset
2000-07-31 21:29:14 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (bitmap_is_set(table->write_set, (*ptr)->field_index) &&
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
(*ptr)->cmp_binary_offset(table->s->rec_buff_length))
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
return TRUE;
Import changeset
2000-07-31 21:29:14 +02:00
}
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
return FALSE;
Import changeset
2000-07-31 21:29:14 +02:00
}
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
/*
check that all fields are real fields
SYNOPSIS
check_fields()
fixed using VIEW fields (BUG#4617)
2004-07-23 08:20:58 +02:00
thd thread handler
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
items Items for check
RETURN
TRUE Items can't be used in UPDATE
FALSE Items are OK
*/
fixed using VIEW fields (BUG#4617)
2004-07-23 08:20:58 +02:00
static bool check_fields(THD *thd, List<Item> &items)
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
{
fixed using VIEW fields (BUG#4617)
2004-07-23 08:20:58 +02:00
List_iterator<Item> it(items);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
Item *item;
updateability of field with collation support (BUG#5506)
2004-09-16 22:45:20 +02:00
Item_field *field;
Name resolution context added (BUG#6443)
2005-07-01 06:05:42 +02:00
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
while ((item= it++))
{
updateability of field with collation support (BUG#5506)
2004-09-16 22:45:20 +02:00
if (!(field= item->filed_for_view_update()))
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
{
fixed & added comments (according to Igor's review and a bit more)
2004-10-07 00:45:06 +02:00
/* item has name, because it comes from VIEW SELECT list */
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
my_error(ER_NONUPDATEABLE_COLUMN, MYF(0), item->name);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
return TRUE;
}
fixed using VIEW fields (BUG#4617)
2004-07-23 08:20:58 +02:00
/*
we make temporary copy of Item_field, to avoid influence of changing
result_field on Item_ref which refer on this field
*/
Fixes after merge with 4.1 FOUND is not a reserved keyword anymore Added Item_field::set_no_const_sub() to be able to mark fields that can't be substituted Added 'simple_select' method to be able to quickly determinate if a select_result is a normal SELECT Note that the 5.0 tree is not yet up to date: Sanja will have to fix multi-update-locks for this merge to be complete
2004-11-03 11:39:38 +01:00
thd->change_item_tree(it.ref(), new Item_field(thd, field));
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
}
return FALSE;
}
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
/*
Process usual UPDATE
SYNOPSIS
mysql_update()
thd thread handler
fields fields for update
values values of fields for update
conds WHERE clause expression
order_num number of elemen in ORDER BY clause
order ORDER BY clause list
limit limit clause
handle_duplicates how to handle duplicates
RETURN
0 - OK
2 - privilege check and openning table passed, but we need to convert to
multi-update because of view substitution
merge
2004-11-21 19:08:12 +01:00
1 - error
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
*/
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
int mysql_update(THD *thd,
TABLE_LIST *table_list,
List<Item> &fields,
List<Item> &values,
COND *conds,
fixed subselects with temporary tables (SCRUM) fixed memory leacks
2003-01-25 01:25:52 +01:00
uint order_num, ORDER *order,
Import changeset
2000-07-31 21:29:14 +02:00
ha_rows limit,
Merge with 4.1 tree to get fix for INSERT IGNORE ... ON DUPLICATE KEY
2005-01-03 22:04:52 +01:00
enum enum_duplicates handle_duplicates, bool ignore)
Import changeset
2000-07-31 21:29:14 +02:00
{
post review changes: CHECK OPTION moved to one function view name added to error messages
2004-09-29 15:35:01 +02:00
bool using_limit= limit != HA_POS_ERROR;
Fixed compiler warnings (Mostly VC++): - Removed not used variables - Changed some ulong parameters/variables to ulonglong (possible serious bug) - Added casts to get rid of safe assignment from longlong to long (and similar) - Added casts to function parameters - Fixed signed/unsigned compares - Added some constructores to structures - Removed some not portable constructs Better fix for bug Bug #21428 "skipped 9 bytes from file: socket (3)" on "mysqladmin shutdown" (Added new parameter to net_clear() to define when we want the communication buffer to be emptied)
2006-11-30 02:40:42 +01:00
bool safe_update= test(thd->options & OPTION_SAFE_UPDATES);
wl2445.patch
2005-05-07 14:06:07 +02:00
bool used_key_is_modified, transactional_table, will_batch;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
bool can_compare_record;
post review changes: CHECK OPTION moved to one function view name added to error messages
2004-09-29 15:35:01 +02:00
int res;
Merge mysql.com:/home/my/mysql-5.0 into mysql.com:/home/my/mysql-5.1
2005-11-05 00:32:55 +01:00
int error, loc_error;
uint used_index= MAX_KEY, dup_key_found;
BUG#12915: Added single-table UPDATE/DELTE ... ORDER BY ... LIMIT optimization: now can use index to find records to update/delete when there is no WHERE clause.
2005-09-30 13:21:37 +02:00
bool need_sort= TRUE;
New handler::index_flags() definition to make it easy to check the full used key and a specific key part. Added key part to optimize_range() to fix problems when using fields in key parts.
2004-07-08 14:45:25 +02:00
#ifndef NO_EMBEDDED_ACCESS_CHECKS
uint want_privilege;
#endif
post merge
2004-11-21 18:33:49 +01:00
uint table_count= 0;
Fix problem where key_read was not cleaned up properly, which caused assert in innodb test.
2003-05-03 15:21:43 +02:00
ha_rows updated, found;
Import changeset
2000-07-31 21:29:14 +02:00
key_map old_used_keys;
TABLE *table;
SQL_SELECT *select;
READ_RECORD info;
Few more places with thd->lex replaced with lex (mysql_update, check_simple_select). This change was inspired by Monty in December 2003 when thd->lex became a pointer in 4.1.
2004-11-11 20:37:48 +01:00
SELECT_LEX *select_lex= &thd->lex->select_lex;
WL#3146 "less locking in auto_increment": this is a cleanup patch for our current auto_increment handling: new names for auto_increment variables in THD, new methods to manipulate them (see sql_class.h), some move into handler::, causing less backup/restore work when executing substatements. This makes the logic hopefully clearer, less work is is needed in mysql_insert(). By cleaning up, using different variables for different purposes (instead of one for 3 things...), we fix those bugs, which someone may want to fix in 5.0 too: BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate statement-based" BUG#20341 "stored function inserting into one auto_increment puts bad data in slave" BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE" (now if a row is updated, LAST_INSERT_ID() will return its id) and re-fixes: BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT" (already fixed differently by Ramil in 4.1) Test of documented behaviour of mysql_insert_id() (there was no test). The behaviour changes introduced are: - LAST_INSERT_ID() now returns "the first autogenerated auto_increment value successfully inserted", instead of "the first autogenerated auto_increment value if any row was successfully inserted", see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY UPDATE, see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() does not change if no autogenerated value was successfully inserted (it used to then be 0), see auto_increment.test. - if in INSERT SELECT no autogenerated value was successfully inserted, mysql_insert_id() now returns the id of the last inserted row (it already did this for INSERT VALUES), see mysql_client_test.c. - if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X (it already did this for INSERT VALUES), see mysql_client_test.c. - NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE, the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID influences not only the first row now. Additionally, when unlocking a table we check that the thread is not keeping a next_insert_id (as the table is unlocked that id is potentially out-of-date); forgetting about this next_insert_id is done in a new handler::ha_release_auto_increment(). Finally we prepare for engines capable of reserving finite-length intervals of auto_increment values: we store such intervals in THD. The next step (to be done by the replication team in 5.1) is to read those intervals from THD and actually store them in the statement-based binary log. NDB will be a good engine to test that.
2006-07-09 17:52:19 +02:00
bool need_reopen;
ulonglong id;
Import changeset
2000-07-31 21:29:14 +02:00
DBUG_ENTER("mysql_update");
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
for ( ; ; )
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
{
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
if (open_tables(thd, &table_list, &table_count, 0))
DBUG_RETURN(1);
if (table_list->multitable_view)
{
DBUG_ASSERT(table_list->view != 0);
DBUG_PRINT("info", ("Switch to multi-update"));
/* pass counter value */
thd->lex->table_count= table_count;
/* convert to multiupdate */
DBUG_RETURN(2);
}
if (!lock_tables(thd, table_list, table_count, &need_reopen))
break;
if (!need_reopen)
DBUG_RETURN(1);
Fix for bug #16593 "Deadlock or crash in stress test for case where trigger starts trigger". In short, the deadlock/crash happened when execution of statement, which used stored functions or activated triggers, coincided with alteration of the tables used by these functions or triggers (in highly concurrent environment). Bug was caused by the incorrect handling of tables from prelocked set in open_tables() functions in situations when refresh happened. This fix replaces old smart but not very robust way of handling tables after refresh (which was closing only old tables), with new one which simply closes all tables opened so far and restarts open_tables(). Also fixed handling of temporary tables in close_tables_for_reopen(). No test case present since bug manifests itself only in concurrent environment.
2006-02-16 14:19:24 +01:00
close_tables_for_reopen(thd, &table_list);
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
}
post merge
2004-11-21 18:33:49 +01:00
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
if (mysql_handle_derived(thd->lex, &mysql_derived_prepare) ||
post merge
2004-11-21 18:33:49 +01:00
(thd->fill_derived_tables() &&
mysql_handle_derived(thd->lex, &mysql_derived_filling)))
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(1);
post merge
2004-11-21 18:33:49 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
thd->proc_info="init";
merging
2002-11-23 18:20:04 +01:00
table= table_list->table;
Import changeset
2000-07-31 21:29:14 +02:00
Added Innobase to source distribution
2001-02-17 13:19:19 +01:00
/* Calculate "table->used_keys" based on the WHERE */
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
table->used_keys= table->s->keys_in_use;
key_map with more that 64 bits width
2003-10-11 13:06:55 +02:00
table->quick_keys.clear_all();
after review PS fixes
2004-04-10 00:14:32 +02:00
SCRUM: WL#604 Privileges in embedded library code added to check privileges in embedded library NO_EMBEDDED_ACCESS_CHECKS macros inserted in code so we can exclude access-checking parts. Actually we now can exclude these parts from standalone server as well. Do we need it? Access checks are disabled in embedded server by default. One should edit libmysqld/Makefile manually to get this working. We definitely need the separate configure for embedded server
2003-09-26 12:33:13 +02:00
#ifndef NO_EMBEDDED_ACCESS_CHECKS
fixed bugs in view code with prepared statemnts
2005-01-04 17:04:16 +01:00
/* TABLE_LIST contain right privilages request */
want_privilege= table_list->grant.want_privilege;
SCRUM: WL#604 Privileges in embedded library code added to check privileges in embedded library NO_EMBEDDED_ACCESS_CHECKS macros inserted in code so we can exclude access-checking parts. Actually we now can exclude these parts from standalone server as well. Do we need it? Access checks are disabled in embedded server by default. One should edit libmysqld/Makefile manually to get this working. We definitely need the separate configure for embedded server
2003-09-26 12:33:13 +02:00
#endif
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
if (mysql_prepare_update(thd, table_list, &conds, order_num, order))
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(1);
check of using same table for changing and select
2002-11-30 18:26:18 +01:00
Merge with 4.0.13
2003-05-19 15:35:49 +02:00
old_used_keys= table->used_keys; // Keys used in WHERE
Added Innobase to source distribution
2001-02-17 13:19:19 +01:00
/* Check the fields we are going to modify */
SCRUM: WL#604 Privileges in embedded library code added to check privileges in embedded library NO_EMBEDDED_ACCESS_CHECKS macros inserted in code so we can exclude access-checking parts. Actually we now can exclude these parts from standalone server as well. Do we need it? Access checks are disabled in embedded server by default. One should edit libmysqld/Makefile manually to get this working. We definitely need the separate configure for embedded server
2003-09-26 12:33:13 +02:00
#ifndef NO_EMBEDDED_ACCESS_CHECKS
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
table_list->grant.want_privilege= table->grant.want_privilege= want_privilege;
support of view underlying tables and SP functions security check added (BUG#9505) (WL#2787)
2005-10-27 23:18:23 +02:00
table_list->register_want_access(want_privilege);
SCRUM: WL#604 Privileges in embedded library code added to check privileges in embedded library NO_EMBEDDED_ACCESS_CHECKS macros inserted in code so we can exclude access-checking parts. Actually we now can exclude these parts from standalone server as well. Do we need it? Access checks are disabled in embedded server by default. One should edit libmysqld/Makefile manually to get this working. We definitely need the separate configure for embedded server
2003-09-26 12:33:13 +02:00
#endif
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (setup_fields_with_no_wrap(thd, 0, fields, MARK_COLUMNS_WRITE, 0, 0))
Name resolution context added (BUG#6443)
2005-07-01 06:05:42 +02:00
DBUG_RETURN(1); /* purecov: inspected */
fixed using VIEW fields (BUG#4617)
2004-07-23 08:20:58 +02:00
if (table_list->view && check_fields(thd, fields))
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
{
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(1);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
}
if (!table_list->updatable || check_key_in_view(thd, table_list))
{
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
my_error(ER_NON_UPDATABLE_TABLE, MYF(0), table_list->alias, "UPDATE");
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(1);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
}
Added Innobase to source distribution
2001-02-17 13:19:19 +01:00
if (table->timestamp_field)
{
// Don't set timestamp column if this is modified
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (bitmap_is_set(table->write_set,
table->timestamp_field->field_index))
Support for TIMESTAMP columns holding NULL values. Unlike all other column types TIMESTAMP is NOT NULL by default, so in order to have TIMESTAMP column holding NULL valaues you have to specify NULL as one of its attributes (this needed for backward compatibility). Main changes: Replaced TABLE::timestamp_default_now/on_update_now members with TABLE::timestamp_auto_set_type flag which is used everywhere for determining if we should auto-set value of TIMESTAMP field during this operation or not. We are also use Field_timestamp::set_time() instead of handler::update_timestamp() in handlers.
2004-10-01 16:54:06 +02:00
table->timestamp_field_type= TIMESTAMP_NO_AUTO_SET;
Added Innobase to source distribution
2001-02-17 13:19:19 +01:00
else
WL 1682: After review fixes + update bitvector class + bitvector test cases + bug fixes
2005-05-02 15:45:33 +02:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
bitmap_set_bit(table->write_set,
table->timestamp_field->field_index);
WL 1682: After review fixes + update bitvector class + bitvector test cases + bug fixes
2005-05-02 15:45:33 +02:00
}
Added Innobase to source distribution
2001-02-17 13:19:19 +01:00
}
Bugfix when using a multi-part unique key in the SET and WHERE part Put PRIMARY KEY and UNIQUE keys before other keys Fixes for auto_increment keys for BDB tables
2000-12-10 20:10:03 +01:00
SCRUM: WL#604 Privileges in embedded library code added to check privileges in embedded library NO_EMBEDDED_ACCESS_CHECKS macros inserted in code so we can exclude access-checking parts. Actually we now can exclude these parts from standalone server as well. Do we need it? Access checks are disabled in embedded server by default. One should edit libmysqld/Makefile manually to get this working. We definitely need the separate configure for embedded server
2003-09-26 12:33:13 +02:00
#ifndef NO_EMBEDDED_ACCESS_CHECKS
Added Innobase to source distribution
2001-02-17 13:19:19 +01:00
/* Check values */
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
table_list->grant.want_privilege= table->grant.want_privilege=
Fixed two bugs in MySQL ACL. First one is related to Bug#7905. One should not be allowed to create new user with password without UPDATE privilege to MySQL database. Furthermore, executing the same GRANT statement twice would actually crash the server and corrupt privilege database. Other bug was that one could update a column, using the existing value as basis to calculate the new value (e.g. UPDATE t1 SET a=a+1) without SELECT privilege to the field (a in the above example) Fixed tests grant.pl and grant2, which were wrong.
2005-03-17 07:16:56 +01:00
(SELECT_ACL & ~table->grant.privilege);
SCRUM: WL#604 Privileges in embedded library code added to check privileges in embedded library NO_EMBEDDED_ACCESS_CHECKS macros inserted in code so we can exclude access-checking parts. Actually we now can exclude these parts from standalone server as well. Do we need it? Access checks are disabled in embedded server by default. One should edit libmysqld/Makefile manually to get this working. We definitely need the separate configure for embedded server
2003-09-26 12:33:13 +02:00
#endif
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (setup_fields(thd, 0, values, MARK_COLUMNS_READ, 0, 0))
Import changeset
2000-07-31 21:29:14 +02:00
{
Few more places with thd->lex replaced with lex (mysql_update, check_simple_select). This change was inspired by Monty in December 2003 when thd->lex became a pointer in 4.1.
2004-11-11 20:37:48 +01:00
free_underlaid_joins(thd, select_lex);
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(1); /* purecov: inspected */
Import changeset
2000-07-31 21:29:14 +02:00
}
Removed init that crashed BDB Fixed newly introduced bug in update
2000-12-11 01:31:09 +01:00
Fix bug#14186 select datefield is null not updated Date field was declared as not null, thus expression 'datefield is null' was always false. For SELECT special handling of such cases is used. There 'datefield is null' converted to 'datefield eq "0000-00-00"'. In mysql_update() before creation of select added remove_eq_conds() call. It makes some optimization of conds and in particular performs conversion from 'is null' to 'eq'. Also remove_eq_conds() makes some evaluation of conds and if it founds that conds is always false then update statement is not processed further. All this allows to perform some update statements process faster due to optimized conds, and not wasting resources if conds known to be false.
2005-10-27 23:24:11 +02:00
if (conds)
{
Item::cond_result cond_value;
conds= remove_eq_conds(thd, conds, &cond_value);
if (cond_value == Item::COND_FALSE)
limit= 0; // Impossible WHERE
}
Bugfix when using a multi-part unique key in the SET and WHERE part Put PRIMARY KEY and UNIQUE keys before other keys Fixes for auto_increment keys for BDB tables
2000-12-10 20:10:03 +01:00
// Don't count on usage of 'only index' when calculating which key to use
key_map with more that 64 bits width
2003-10-11 13:06:55 +02:00
table->used_keys.clear_all();
WL#2985 "Partition pruning": addition2: perform partition pruning for single-table UPDATE/DELETE.
2006-01-26 18:17:17 +01:00
#ifdef WITH_PARTITION_STORAGE_ENGINE
if (prune_partitions(thd, table, conds))
{
free_underlaid_joins(thd, select_lex);
send_ok(thd); // No matching records
DBUG_RETURN(0);
}
#endif
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
/* Update the table->file->stats.records number */
WL#2985 "Partition pruning", "do pruning for UPDATE/DELETE": Post-merge fixes
2006-02-06 16:33:39 +01:00
table->file->info(HA_STATUS_VARIABLE | HA_STATUS_NO_LOCK);
WL#2985 "Partition pruning": addition2: perform partition pruning for single-table UPDATE/DELETE.
2006-01-26 18:17:17 +01:00
Cleanup during reviews Removed some optional arguments Fixed portability problem in federated tests
2005-03-16 15:11:01 +01:00
select= make_select(table, 0, 0, conds, 0, &error);
Fix bug#14186 select datefield is null not updated Date field was declared as not null, thus expression 'datefield is null' was always false. For SELECT special handling of such cases is used. There 'datefield is null' converted to 'datefield eq "0000-00-00"'. In mysql_update() before creation of select added remove_eq_conds() call. It makes some optimization of conds and in particular performs conversion from 'is null' to 'eq'. Also remove_eq_conds() makes some evaluation of conds and if it founds that conds is always false then update statement is not processed further. All this allows to perform some update statements process faster due to optimized conds, and not wasting resources if conds known to be false.
2005-10-27 23:24:11 +02:00
if (error || !limit ||
(select && select->check_quick(thd, safe_update, limit)))
Import changeset
2000-07-31 21:29:14 +02:00
{
delete select;
Few more places with thd->lex replaced with lex (mysql_update, check_simple_select). This change was inspired by Monty in December 2003 when thd->lex became a pointer in 4.1.
2004-11-11 20:37:48 +01:00
free_underlaid_joins(thd, select_lex);
Import changeset
2000-07-31 21:29:14 +02:00
if (error)
{
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(1); // Error in where
Import changeset
2000-07-31 21:29:14 +02:00
}
Fixes and code cleanups after merge with 4.0.3 Warning handling and initial prepared statement handling (last not complete yet) Changed a lot of functions that returned 0/1 to my_bool type. GRANT handling now uses read/write locks instead of mutex Change basic net functions to use THD instead of NET (needed for 4.1 protocol) Use my_sprintf instead of sprintf() + strlen() Added alloc_query() to be able to chare query initialization code with prepared statements. Cleanup handling of SHOW COUNT(*) WARNINGS and SELECT LAST_INSERT_ID() Note that the following test fails (will be fixed ASAP): sub_select, union, rpl_rotate_logs and rpl_mystery22
2002-10-02 12:33:08 +02:00
send_ok(thd); // No matching records
Import changeset
2000-07-31 21:29:14 +02:00
DBUG_RETURN(0);
}
BUG#12915: Added single-table UPDATE/DELTE ... ORDER BY ... LIMIT optimization: now can use index to find records to update/delete when there is no WHERE clause.
2005-09-30 13:21:37 +02:00
if (!select && limit != HA_POS_ERROR)
{
Added more tests for new UPDATE ... ORDER BY ... LIMIT optimization
2005-10-25 01:27:40 +02:00
if ((used_index= get_index_for_order(table, order, limit)) != MAX_KEY)
BUG#12915: Added single-table UPDATE/DELTE ... ORDER BY ... LIMIT optimization: now can use index to find records to update/delete when there is no WHERE clause.
2005-09-30 13:21:37 +02:00
need_sort= FALSE;
}
Import changeset
2000-07-31 21:29:14 +02:00
/* If running in safe sql mode, don't allow updates without keys */
key_map with more that 64 bits width
2003-10-11 13:06:55 +02:00
if (table->quick_keys.is_clear_all())
Import changeset
2000-07-31 21:29:14 +02:00
{
QUERY_NO_GOOD_INDEX_USED and QUERY_NO_INDEX_USED moved from thd->lex.select_lex->options to thd->server_status
2003-12-06 23:21:09 +01:00
thd->server_status|=SERVER_QUERY_NO_INDEX_USED;
A fix for safe updates
2002-11-15 21:01:28 +01:00
if (safe_update && !using_limit)
Bug fix for NATURAL JOIN, Split manual sections and more statistic variables
2000-11-20 01:57:02 +01:00
{
Merge with 4.0.13
2003-05-19 15:35:49 +02:00
my_message(ER_UPDATE_WITHOUT_KEY_IN_SAFE_MODE,
ER(ER_UPDATE_WITHOUT_KEY_IN_SAFE_MODE), MYF(0));
goto err;
Bug fix for NATURAL JOIN, Split manual sections and more statistic variables
2000-11-20 01:57:02 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
}
Few more places with thd->lex replaced with lex (mysql_update, check_simple_select). This change was inspired by Monty in December 2003 when thd->lex became a pointer in 4.1.
2004-11-11 20:37:48 +01:00
init_ftfuncs(thd, select_lex, 1);
Fix for bug#20670 "UPDATE using key and invoking trigger that modifies this key does not stop" (5.1 version). UPDATE statement which WHERE clause used key and which invoked trigger that modified field in this key worked indefinetely. This problem occured because in cases when UPDATE statement was executed in update-on-the-fly mode (in which row is updated right during evaluation of select for WHERE clause) the new version of the row became visible to select representing WHERE clause and was updated again and again. We already solve this problem for UPDATE statements which does not invoke triggers by detecting the fact that we are going to update field in key used for scanning and performing update in two steps, during the first step we gather information about the rows to be updated and then doing actual updates. We also do this for MULTI-UPDATE and in its case we even detect situation when such fields are updated in triggers (actually we simply assume that we always update fields used in key if we have before update trigger). The fix simply extends this check which is done with help of check_if_key_used()/QUICK_SELECT_I::check_if_keys_used() routine/method in such way that it also detects cases when field used in key is updated in trigger. We do this by changing check_if_key_used() to take field bitmap instead field list as argument and passing TABLE::write_set to it (we also have to add info about fields used in triggers to this bitmap a bit earlier). As nice side-effect we have more precise and thus more optimal perfomance-wise check for the MULTI-UPDATE. Also check_if_key_used() routine and similar method were renamed to is_key_used()/is_keys_used() in order to better reflect that it is simple boolean predicate. Finally, partition_key_modified() routine now also takes field bitmap instead of field list as argument.
2006-09-21 11:39:29 +02:00
table->mark_columns_needed_for_update();
Import changeset
2000-07-31 21:29:14 +02:00
/* Check if we are modifying a key that we are used to search with */
This is first cset for WL#1394 "Optimize index merge when all involved index ranges include only values with equal keys" The main idea is to exploit the fact that key scans for "key=const" return ordered sequences of rowids.
2004-05-12 23:38:40 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
if (select && select->quick)
Added index_merge access method
2003-11-13 15:52:02 +01:00
{
More code cleanup, debug prints removed Fixed an incorrect optimizer choice for ror-intersect(key, clustered_primary) Typo bug fixed in multitable update
2004-06-01 13:31:58 +02:00
used_index= select->quick->index;
This is first cset for WL#1394 "Optimize index merge when all involved index ranges include only values with equal keys" The main idea is to exploit the fact that key scans for "key=const" return ordered sequences of rowids.
2004-05-12 23:38:40 +02:00
used_key_is_modified= (!select->quick->unique_key_range() &&
Fix for bug#20670 "UPDATE using key and invoking trigger that modifies this key does not stop" (5.1 version). UPDATE statement which WHERE clause used key and which invoked trigger that modified field in this key worked indefinetely. This problem occured because in cases when UPDATE statement was executed in update-on-the-fly mode (in which row is updated right during evaluation of select for WHERE clause) the new version of the row became visible to select representing WHERE clause and was updated again and again. We already solve this problem for UPDATE statements which does not invoke triggers by detecting the fact that we are going to update field in key used for scanning and performing update in two steps, during the first step we gather information about the rows to be updated and then doing actual updates. We also do this for MULTI-UPDATE and in its case we even detect situation when such fields are updated in triggers (actually we simply assume that we always update fields used in key if we have before update trigger). The fix simply extends this check which is done with help of check_if_key_used()/QUICK_SELECT_I::check_if_keys_used() routine/method in such way that it also detects cases when field used in key is updated in trigger. We do this by changing check_if_key_used() to take field bitmap instead field list as argument and passing TABLE::write_set to it (we also have to add info about fields used in triggers to this bitmap a bit earlier). As nice side-effect we have more precise and thus more optimal perfomance-wise check for the MULTI-UPDATE. Also check_if_key_used() routine and similar method were renamed to is_key_used()/is_keys_used() in order to better reflect that it is simple boolean predicate. Finally, partition_key_modified() routine now also takes field bitmap instead of field list as argument.
2006-09-21 11:39:29 +02:00
select->quick->is_keys_used(table->write_set));
Added index_merge access method
2003-11-13 15:52:02 +01:00
}
Import changeset
2000-07-31 21:29:14 +02:00
else
BUG#12915: Added single-table UPDATE/DELTE ... ORDER BY ... LIMIT optimization: now can use index to find records to update/delete when there is no WHERE clause.
2005-09-30 13:21:37 +02:00
{
Added more tests for new UPDATE ... ORDER BY ... LIMIT optimization
2005-10-25 01:27:40 +02:00
used_key_is_modified= 0;
if (used_index == MAX_KEY) // no index for sort order
used_index= table->file->key_used_on_scan;
if (used_index != MAX_KEY)
Fix for bug#20670 "UPDATE using key and invoking trigger that modifies this key does not stop" (5.1 version). UPDATE statement which WHERE clause used key and which invoked trigger that modified field in this key worked indefinetely. This problem occured because in cases when UPDATE statement was executed in update-on-the-fly mode (in which row is updated right during evaluation of select for WHERE clause) the new version of the row became visible to select representing WHERE clause and was updated again and again. We already solve this problem for UPDATE statements which does not invoke triggers by detecting the fact that we are going to update field in key used for scanning and performing update in two steps, during the first step we gather information about the rows to be updated and then doing actual updates. We also do this for MULTI-UPDATE and in its case we even detect situation when such fields are updated in triggers (actually we simply assume that we always update fields used in key if we have before update trigger). The fix simply extends this check which is done with help of check_if_key_used()/QUICK_SELECT_I::check_if_keys_used() routine/method in such way that it also detects cases when field used in key is updated in trigger. We do this by changing check_if_key_used() to take field bitmap instead field list as argument and passing TABLE::write_set to it (we also have to add info about fields used in triggers to this bitmap a bit earlier). As nice side-effect we have more precise and thus more optimal perfomance-wise check for the MULTI-UPDATE. Also check_if_key_used() routine and similar method were renamed to is_key_used()/is_keys_used() in order to better reflect that it is simple boolean predicate. Finally, partition_key_modified() routine now also takes field bitmap instead of field list as argument.
2006-09-21 11:39:29 +02:00
used_key_is_modified= is_key_used(table, used_index, table->write_set);
BUG#12915: Added single-table UPDATE/DELTE ... ORDER BY ... LIMIT optimization: now can use index to find records to update/delete when there is no WHERE clause.
2005-09-30 13:21:37 +02:00
}
Fixed failing test cases 'row.test' when running with --ps-protocol Simple optimzations done while reviewing code
2005-02-15 15:45:00 +01:00
Fix for bug#20670 "UPDATE using key and invoking trigger that modifies this key does not stop" (5.1 version). UPDATE statement which WHERE clause used key and which invoked trigger that modified field in this key worked indefinetely. This problem occured because in cases when UPDATE statement was executed in update-on-the-fly mode (in which row is updated right during evaluation of select for WHERE clause) the new version of the row became visible to select representing WHERE clause and was updated again and again. We already solve this problem for UPDATE statements which does not invoke triggers by detecting the fact that we are going to update field in key used for scanning and performing update in two steps, during the first step we gather information about the rows to be updated and then doing actual updates. We also do this for MULTI-UPDATE and in its case we even detect situation when such fields are updated in triggers (actually we simply assume that we always update fields used in key if we have before update trigger). The fix simply extends this check which is done with help of check_if_key_used()/QUICK_SELECT_I::check_if_keys_used() routine/method in such way that it also detects cases when field used in key is updated in trigger. We do this by changing check_if_key_used() to take field bitmap instead field list as argument and passing TABLE::write_set to it (we also have to add info about fields used in triggers to this bitmap a bit earlier). As nice side-effect we have more precise and thus more optimal perfomance-wise check for the MULTI-UPDATE. Also check_if_key_used() routine and similar method were renamed to is_key_used()/is_keys_used() in order to better reflect that it is simple boolean predicate. Finally, partition_key_modified() routine now also takes field bitmap instead of field list as argument.
2006-09-21 11:39:29 +02:00
Make storage engines "pluggable", handlerton work
2005-11-07 16:25:06 +01:00
#ifdef WITH_PARTITION_STORAGE_ENGINE
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
if (used_key_is_modified || order ||
Fix for bug#20670 "UPDATE using key and invoking trigger that modifies this key does not stop" (5.1 version). UPDATE statement which WHERE clause used key and which invoked trigger that modified field in this key worked indefinetely. This problem occured because in cases when UPDATE statement was executed in update-on-the-fly mode (in which row is updated right during evaluation of select for WHERE clause) the new version of the row became visible to select representing WHERE clause and was updated again and again. We already solve this problem for UPDATE statements which does not invoke triggers by detecting the fact that we are going to update field in key used for scanning and performing update in two steps, during the first step we gather information about the rows to be updated and then doing actual updates. We also do this for MULTI-UPDATE and in its case we even detect situation when such fields are updated in triggers (actually we simply assume that we always update fields used in key if we have before update trigger). The fix simply extends this check which is done with help of check_if_key_used()/QUICK_SELECT_I::check_if_keys_used() routine/method in such way that it also detects cases when field used in key is updated in trigger. We do this by changing check_if_key_used() to take field bitmap instead field list as argument and passing TABLE::write_set to it (we also have to add info about fields used in triggers to this bitmap a bit earlier). As nice side-effect we have more precise and thus more optimal perfomance-wise check for the MULTI-UPDATE. Also check_if_key_used() routine and similar method were renamed to is_key_used()/is_keys_used() in order to better reflect that it is simple boolean predicate. Finally, partition_key_modified() routine now also takes field bitmap instead of field list as argument.
2006-09-21 11:39:29 +02:00
partition_key_modified(table, table->write_set))
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
#else
After review fixes
2005-05-09 10:30:25 +02:00
if (used_key_is_modified || order)
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
#endif
Import changeset
2000-07-31 21:29:14 +02:00
{
/*
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
We can't update table directly; We must first search after all
matching rows before updating the table!
Import changeset
2000-07-31 21:29:14 +02:00
*/
fixes/cleanups according to Coverity report
2005-01-24 15:48:25 +01:00
if (used_index < MAX_KEY && old_used_keys.is_set(used_index))
Import changeset
2000-07-31 21:29:14 +02:00
{
table->key_read=1;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
table->mark_columns_used_by_index(used_index);
}
else
{
table->use_all_columns();
Import changeset
2000-07-31 21:29:14 +02:00
}
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
/* note: We avoid sorting avoid if we sort on the used index */
Added more tests for new UPDATE ... ORDER BY ... LIMIT optimization
2005-10-25 01:27:40 +02:00
if (order && (need_sort || used_key_is_modified))
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
{
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
/*
Doing an ORDER BY; Let filesort find and sort the rows we are going
to update
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
NOTE: filesort will call table->prepare_for_position()
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
*/
Bug#25172: Not checked buffer size leads to a server crash. After fix for bug#21798 JOIN stores the pointer to the buffer for sorting fields. It is used while sorting for grouping and for ordering. If ORDER BY clause has more elements then the GROUP BY clause then a memory overrun occurs. Now the length of the ORDER BY list is always passed to the make_unireg_sortorder() function and it allocates buffer big enough to be used for bigger list.
2007-01-19 16:34:09 +01:00
uint length= 0;
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
SORT_FIELD *sortorder;
fixes after merge. Now it compiles and passes the test suite
2001-05-11 01:08:29 +02:00
ha_rows examined_rows;
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
Many files: Added the MAX_LENGTH_FOR_SORT_DATA option filesort.cc: Added the MAX_LENGTH_FOR_SORT_DATA option
2003-04-24 13:33:33 +02:00
table->sort.io_cache = (IO_CACHE *) my_malloc(sizeof(IO_CACHE),
Merge with 4.0.13
2003-05-19 15:35:49 +02:00
MYF(MY_FAE | MY_ZEROFILL));
Bug#21798: memory leak during query execution with subquery in column list using a function When executing dependent subqueries they are re-inited and re-exec() for each row of the outer context. The cause for the bug is that during subquery reinitialization/re-execution, the optimizer reallocates JOIN::join_tab, JOIN::table in make_simple_join() and the local variable in 'sortorder' in create_sort_index(), which is allocated by make_unireg_sortorder(). Care must be taken not to allocate anything into the thread's memory pool while re-initializing query plan structures between subquery re-executions. All such items mush be cached and reused because the thread's memory pool is freed at the end of the whole query. Note that they must be cached and reused even for queries that are not otherwise cacheable because otherwise it will grow the thread's memory pool every time a cacheable query is re-executed. We provide additional members to the JOIN structure to store references to the items that need to be cached.
2006-10-17 15:20:26 +02:00
if (!(sortorder=make_unireg_sortorder(order, &length, NULL)) ||
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
(table->sort.found_records= filesort(thd, table, sortorder, length,
select, limit, 1,
&examined_rows))
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
== HA_POS_ERROR)
{
Fix problem where key_read was not cleaned up properly, which caused assert in innodb test.
2003-05-03 15:21:43 +02:00
goto err;
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
}
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
/*
Filesort has already found and selected the rows we want to update,
so we don't need the where clause
*/
delete select;
select= 0;
Added all changes from old 4.0 version: PSTACK, libmysqld and MySQL filesystem UPDATE ... ORDER BY DELETE ... ORDER BY New faster fulltext handling Faster compressed keys
2001-04-11 13:04:03 +02:00
}
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
else
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
/*
We are doing a search on a key that is updated. In this case
we go trough the matching rows, save a pointer to them and
update these in a separate loop based on the pointer.
*/
IO_CACHE tempfile;
if (open_cached_file(&tempfile, mysql_tmpdir,TEMP_PREFIX,
DISK_BUFFER_SIZE, MYF(MY_WME)))
Fix problem where key_read was not cleaned up properly, which caused assert in innodb test.
2003-05-03 15:21:43 +02:00
goto err;
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
Fix for Bug#3183
2004-03-15 21:11:58 +01:00
/* If quick select is used, initialize it before retrieving rows. */
if (select && select->quick && select->quick->reset())
goto err;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
table->file->try_semi_consistent_read(1);
Merge mysql.com:/opt/local/work/mysql-4.1-root into mysql.com:/opt/local/work/mysql-5.0-root
2006-02-02 16:17:18 +01:00
BUG#15935: post-review fixes: added comment
2006-01-25 22:09:04 +01:00
/*
When we get here, we have one of the following options:
A. used_index == MAX_KEY
This means we should use full table scan, and start it with
init_read_record call
B. used_index != MAX_KEY
B.1 quick select is used, start the scan with init_read_record
B.2 quick select is not used, this is full index scan (with LIMIT)
Full index scan must be started with init_read_record_idx
*/
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
Fix for BUG#14272: Don't run index scan when we should use quick select. This could cause failures because there are table handlers (like federated) that support quick select scanning but do not support index scanning.
2005-10-25 14:34:03 +02:00
if (used_index == MAX_KEY || (select && select->quick))
BUG#12915: Added single-table UPDATE/DELTE ... ORDER BY ... LIMIT optimization: now can use index to find records to update/delete when there is no WHERE clause.
2005-09-30 13:21:37 +02:00
init_read_record(&info,thd,table,select,0,1);
else
init_read_record_idx(&info, thd, table, 1, used_index);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
thd->proc_info="Searching rows for update";
uint tmp_limit= limit;
Merge with 4.0.13
2003-05-19 15:35:49 +02:00
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
while (!(error=info.read_record(&info)) && !thd->killed)
Import changeset
2000-07-31 21:29:14 +02:00
{
just tried to find all 'skipp' and replace it with 'skip'.
2004-02-02 17:25:39 +01:00
if (!(select && select->skip_record()))
Import changeset
2000-07-31 21:29:14 +02:00
{
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
if (table->file->was_semi_consistent_read())
continue; /* repeat the read of the same row if it still exists */
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
table->file->position(table->record[0]);
if (my_b_write(&tempfile,table->file->ref,
table->file->ref_length))
{
error=1; /* purecov: inspected */
break; /* purecov: inspected */
}
if (!--limit && using_limit)
Fixed 'Unknown error' when using 'UPDATE ... LIMIT'. Bug 373
2003-05-13 19:07:43 +02:00
{
error= -1;
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
break;
Fixed 'Unknown error' when using 'UPDATE ... LIMIT'. Bug 373
2003-05-13 19:07:43 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
}
sql_update.cc: mysql_update(): Call handler::unlock_row() also in the first while loop. This patch is from Heikki, and it was approved by Sergei Golubchik.
2005-07-13 15:13:36 +02:00
else
table->file->unlock_row();
Import changeset
2000-07-31 21:29:14 +02:00
}
Rollback UPDATE/DELETE statements on kill nsure that rows in a multi-row INSERT DELAYED are inserted atomicly
2004-03-04 17:16:10 +01:00
if (thd->killed && !error)
error= 1; // Aborted
Fixed 'Unknown error' when using 'UPDATE ... LIMIT'. Bug 373
2003-05-13 19:07:43 +02:00
limit= tmp_limit;
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
table->file->try_semi_consistent_read(0);
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
end_read_record(&info);
Added more tests for new UPDATE ... ORDER BY ... LIMIT optimization
2005-10-25 01:27:40 +02:00
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
/* Change select to use tempfile */
if (select)
{
delete select->quick;
if (select->free_cond)
delete select->cond;
select->quick=0;
select->cond=0;
}
Import changeset
2000-07-31 21:29:14 +02:00
else
{
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
select= new SQL_SELECT;
select->head=table;
Import changeset
2000-07-31 21:29:14 +02:00
}
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
if (reinit_io_cache(&tempfile,READ_CACHE,0L,0,0))
error=1; /* purecov: inspected */
select->file=tempfile; // Read row ptrs from this file
if (error >= 0)
Fix problem where key_read was not cleaned up properly, which caused assert in innodb test.
2003-05-03 15:21:43 +02:00
goto err;
Import changeset
2000-07-31 21:29:14 +02:00
}
if (table->key_read)
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
table->restore_column_maps_after_mark_index();
Import changeset
2000-07-31 21:29:14 +02:00
}
Remove DUP_IGNORE from enum_duplicates and instead use a separate ignore flag This allows use to use INSERT IGNORE ... ON DUPLICATE ...
2004-12-31 11:04:35 +01:00
if (ignore)
New benchmark test Fixed bug in REPLACE with BDB tables Prepare for write lock on read for BDB Inform the handler when we want to use IGNORE / REPLACE New manual pages
2000-12-24 14:19:00 +01:00
table->file->extra(HA_EXTRA_IGNORE_DUP_KEY);
This is first cset for WL#1394 "Optimize index merge when all involved index ranges include only values with equal keys" The main idea is to exploit the fact that key scans for "key=const" return ordered sequences of rowids.
2004-05-12 23:38:40 +02:00
if (select && select->quick && select->quick->reset())
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
goto err;
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
table->file->try_semi_consistent_read(1);
Import changeset
2000-07-31 21:29:14 +02:00
init_read_record(&info,thd,table,select,0,1);
Fix problem where key_read was not cleaned up properly, which caused assert in innodb test.
2003-05-03 15:21:43 +02:00
updated= found= 0;
Add warnings for single line inserts. To do this I had to convert count_cuted_fields to an enum (to be able to detect setting a NOT NULL field to NULL)
2003-10-11 22:26:39 +02:00
thd->count_cuted_fields= CHECK_FIELD_WARN; /* calc cuted fields */
Import changeset
2000-07-31 21:29:14 +02:00
thd->cuted_fields=0L;
Fix not critical MyISAM bug in locking. Free character sets properly (for embedded). Add missing mutex in HANDLER CLOSE
2001-11-29 20:46:51 +01:00
thd->proc_info="Updating";
Import changeset
2000-07-31 21:29:14 +02:00
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
transactional_table= table->file->has_transactions();
thd->no_trans_update= 0;
After merge fixes Add support for warnings for prepare of prepared statements Fixed test to work with --ps-protocol Fixed some test results
2005-01-04 12:46:53 +01:00
thd->abort_on_warning= test(!ignore &&
More fixes for strict mode: More tests. Better error messages. Fixed bug when checking if we updated all needed columns for INSERT. Give an error if we encounter a wrong float value during parsing. Don't print DEFAULT for columns without a default value in SHOW CREATE/SHOW FIELDS. Fixed UPDATE IGNORE when using STRICT mode.
2004-10-02 21:20:08 +02:00
(thd->variables.sql_mode &
(MODE_STRICT_TRANS_TABLES |
MODE_STRICT_ALL_TABLES)));
After review fixes
2005-05-09 10:30:25 +02:00
will_batch= !table->file->start_bulk_update();
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
/*
We can use compare_record() to optimize away updates if
the table handler is returning all columns OR if
if all updated columns are read
*/
can_compare_record= (!(table->file->ha_table_flags() &
HA_PARTIAL_COLUMN_READ) ||
bitmap_is_subset(table->write_set, table->read_set));
Merge bodhi.local:/opt/local/work/tmp_merge into bodhi.local:/opt/local/work/mysql-5.1-runtime-merge
2006-07-26 21:33:25 +02:00
Import changeset
2000-07-31 21:29:14 +02:00
while (!(error=info.read_record(&info)) && !thd->killed)
{
just tried to find all 'skipp' and replace it with 'skip'.
2004-02-02 17:25:39 +01:00
if (!(select && select->skip_record()))
Import changeset
2000-07-31 21:29:14 +02:00
{
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
if (table->file->was_semi_consistent_read())
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
continue; /* repeat the read of the same row if it still exists */
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
INSERT ... UPDATE ... VALUES()
2003-05-03 01:16:56 +02:00
store_record(table,record[1]);
Fix for bugs: #5860 "Multi-table UPDATE does not activate update triggers" #6812 "Triggers are not activated for INSERT ... SELECT" #8755 "Trigger is not activated by LOAD DATA". This patch also implements proper handling of triggers for special forms of insert like REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. Also now we don't call after trigger in case when we have failed to inserted/update or delete row. Trigger failure should stop statement execution. I have not properly tested handling of errors which happen inside of triggers in this patch, since it is simplier to do this once we will be able to access tables from triggers.
2005-05-24 20:19:33 +02:00
if (fill_record_n_invoke_before_triggers(thd, fields, values, 0,
table->triggers,
TRG_EVENT_UPDATE))
Added locks needed for Innobase Fixed mutex problem when doing automatic repair of MyISAM tables
2001-02-17 23:03:37 +01:00
break; /* purecov: inspected */
post-review fixes
2004-11-12 13:34:00 +01:00
Import changeset
2000-07-31 21:29:14 +02:00
found++;
WL#1218 "Triggers". Some very preliminary version of patch. Mostly needed for Monty for him getting notion what needed for triggers from new .FRM format. Things to be done: - Right placement of trigger's invocations - Right handling of errors in triggers (including transaction rollback) - Support for priviliges - Right handling of DROP/RENAME table (hope that it will be handled automatically with merging of .TRG into .FRM file) - Saving/restoring some information critical for trigger creation and replication with their definitions (e.g. sql_mode, creator, ...) - Replication Already has some known bugs so probably not for general review.
2004-09-07 14:29:46 +02:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (!can_compare_record || compare_record(table))
Import changeset
2000-07-31 21:29:14 +02:00
{
After merge fixes Add support for warnings for prepare of prepared statements Fixed test to work with --ps-protocol Fixed some test results
2005-01-04 12:46:53 +01:00
if ((res= table_list->view_check_option(thd, ignore)) !=
post review changes: CHECK OPTION moved to one function view name added to error messages
2004-09-29 15:35:01 +02:00
VIEW_CHECK_OK)
{
found--;
if (res == VIEW_CHECK_SKIP)
continue;
else if (res == VIEW_CHECK_ERROR)
{
error= 1;
break;
}
}
wl2445.patch
2005-05-07 14:06:07 +02:00
if (will_batch)
{
/*
Typically a batched handler can execute the batched jobs when:
1) When specifically told to do so
2) When it is not a good idea to batch anymore
3) When it is necessary to send batch for other reasons
(One such reason is when READ's must be performed)
1) is covered by exec_bulk_update calls.
2) and 3) is handled by the bulk_update_row method.
bulk_update_row can execute the updates including the one
defined in the bulk_update_row or not including the row
in the call. This is up to the handler implementation and can
vary from call to call.
The dup_key_found reports the number of duplicate keys found
in those updates actually executed. It only reports those if
the extra call with HA_EXTRA_IGNORE_DUP_KEY have been issued.
If this hasn't been issued it returns an error code and can
ignore this number. Thus any handler that implements batching
for UPDATE IGNORE must also handle this extra call properly.
If a duplicate key is found on the record included in this
call then it should be included in the count of dup_key_found
and error should be set to 0 (only if these errors are ignored).
*/
Patch for push of wl1354 Partitioning
2005-07-18 13:31:02 +02:00
error= table->file->bulk_update_row(table->record[1],
table->record[0],
After review fixes
2005-05-09 10:30:25 +02:00
&dup_key_found);
wl2445.patch
2005-05-07 14:06:07 +02:00
limit+= dup_key_found;
After review fixes
2005-05-09 10:30:25 +02:00
updated-= dup_key_found;
wl2445.patch
2005-05-07 14:06:07 +02:00
}
else
{
/* Non-batched update */
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
error= table->file->ha_update_row((byte*) table->record[1],
(byte*) table->record[0]);
wl2445.patch
2005-05-07 14:06:07 +02:00
}
if (!error)
Import changeset
2000-07-31 21:29:14 +02:00
{
updated++;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
thd->no_trans_update= !transactional_table;
Fix for bugs: #5860 "Multi-table UPDATE does not activate update triggers" #6812 "Triggers are not activated for INSERT ... SELECT" #8755 "Trigger is not activated by LOAD DATA". This patch also implements proper handling of triggers for special forms of insert like REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. Also now we don't call after trigger in case when we have failed to inserted/update or delete row. Trigger failure should stop statement execution. I have not properly tested handling of errors which happen inside of triggers in this patch, since it is simplier to do this once we will be able to access tables from triggers.
2005-05-24 20:19:33 +02:00
if (table->triggers &&
table->triggers->process_triggers(thd, TRG_EVENT_UPDATE,
TRG_ACTION_AFTER, TRUE))
{
error= 1;
break;
}
Import changeset
2000-07-31 21:29:14 +02:00
}
BUG#17138: Error in stored procedure due to fatal error when not really a fatal error. New handling of ignore error in place.
2006-06-20 02:56:50 +02:00
else if (!ignore ||
BUG#17138: Crashes in stored procedure Last round of review fixes
2006-07-01 06:01:37 +02:00
table->file->is_fatal_error(error, HA_CHECK_DUP_KEY))
Import changeset
2000-07-31 21:29:14 +02:00
{
Fixed BUG#13729 Stored procedures: packet error after exception handled Don't set thd->is_fatal_error in sql_update for duplicate key errors.
2005-11-23 11:56:53 +01:00
/*
BUG#17138: Error in stored procedure due to fatal error when not really a fatal error. New handling of ignore error in place.
2006-06-20 02:56:50 +02:00
If (ignore && error is ignorable) we don't have to
Fixed BUG#13729 Stored procedures: packet error after exception handled Don't set thd->is_fatal_error in sql_update for duplicate key errors.
2005-11-23 11:56:53 +01:00
do anything; otherwise...
*/
BUG#17138: Crashes in stored procedure Last round of review fixes
2006-07-01 06:01:37 +02:00
if (table->file->is_fatal_error(error, HA_CHECK_DUP_KEY))
Fixed BUG#13729 Stored procedures: packet error after exception handled Don't set thd->is_fatal_error in sql_update for duplicate key errors.
2005-11-23 11:56:53 +01:00
thd->fatal_error(); /* Other handler errors are fatal */
Import changeset
2000-07-31 21:29:14 +02:00
table->file->print_error(error,MYF(0));
error= 1;
break;
}
}
WL#1218 "Triggers". Some very preliminary version of patch. Mostly needed for Monty for him getting notion what needed for triggers from new .FRM format. Things to be done: - Right placement of trigger's invocations - Right handling of errors in triggers (including transaction rollback) - Support for priviliges - Right handling of DROP/RENAME table (hope that it will be handled automatically with merging of .TRG into .FRM file) - Saving/restoring some information critical for trigger creation and replication with their definitions (e.g. sql_mode, creator, ...) - Replication Already has some known bugs so probably not for general review.
2004-09-07 14:29:46 +02:00
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
if (!--limit && using_limit)
{
wl2445.patch
2005-05-07 14:06:07 +02:00
/*
We have reached end-of-file in most common situations where no
batching has occurred and if batching was supposed to occur but
no updates were made and finally when the batch execution was
performed without error and without finding any duplicate keys.
If the batched updates were performed with errors we need to
check and if no error but duplicate key's found we need to
continue since those are not counted for in limit.
*/
if (will_batch &&
((error= table->file->exec_bulk_update(&dup_key_found)) ||
!dup_key_found))
{
if (error)
{
/*
The handler should not report error of duplicate keys if they
are ignored. This is a requirement on batching handlers.
*/
table->file->print_error(error,MYF(0));
After review fixes
2005-05-09 10:30:25 +02:00
error= 1;
wl2445.patch
2005-05-07 14:06:07 +02:00
break;
}
/*
Either an error was found and we are ignoring errors or there
were duplicate keys found. In both cases we need to correct
the counters and continue the loop.
*/
After review fixes
2005-05-09 10:30:25 +02:00
limit= dup_key_found; //limit is 0 when we get here so need to +
updated-= dup_key_found;
wl2445.patch
2005-05-07 14:06:07 +02:00
}
else
{
error= -1; // Simulate end of file
break;
}
Fixes for valgrind Added optimzation for clustered index Fixed bug in UPDATE ... ORDER BY Fixed handling of UPDATE ... LIMIT
2003-04-23 20:52:16 +02:00
}
Import changeset
2000-07-31 21:29:14 +02:00
}
Portability fixes. Patches required by Gemini Fix to properly detect if there is an active transaction in InnoDB Fix to not lock thread structure when doing automatic rollback when thread ends Allow -O lower_case_names=0 on UNIX
2001-08-21 19:06:00 +02:00
else
table->file->unlock_row();
Support of warnings for all DML statements (Insert, Update and Alter) Fix LOAD DATA INFILE warnings to have a better meanigful messages Fix to make the mysql command line to automatically show the warnings count for all basic commands
2003-04-30 09:02:28 +02:00
thd->row_count++;
Import changeset
2000-07-31 21:29:14 +02:00
}
After review fixes
2005-05-09 10:30:25 +02:00
dup_key_found= 0;
Rollback UPDATE/DELETE statements on kill nsure that rows in a multi-row INSERT DELAYED are inserted atomicly
2004-03-04 17:16:10 +01:00
if (thd->killed && !error)
error= 1; // Aborted
wl2445.patch
2005-05-07 14:06:07 +02:00
else if (will_batch &&
(loc_error= table->file->exec_bulk_update(&dup_key_found)))
/*
An error has occurred when a batched update was performed and returned
an error indication. It cannot be an allowed duplicate key error since
we require the batching handler to treat this as a normal behavior.
Otherwise we simply remove the number of duplicate keys records found
in the batched update.
*/
{
thd->fatal_error();
table->file->print_error(loc_error,MYF(0));
After review fixes
2005-05-09 10:30:25 +02:00
error= 1;
wl2445.patch
2005-05-07 14:06:07 +02:00
}
else
After review fixes
2005-05-09 10:30:25 +02:00
updated-= dup_key_found;
wl2445.patch
2005-05-07 14:06:07 +02:00
if (will_batch)
table->file->end_bulk_update();
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
table->file->try_semi_consistent_read(0);
Import changeset
2000-07-31 21:29:14 +02:00
end_read_record(&info);
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
delete select;
After review fixes
2005-05-09 10:30:25 +02:00
thd->proc_info= "end";
New benchmark test Fixed bug in REPLACE with BDB tables Prepare for write lock on read for BDB Inform the handler when we want to use IGNORE / REPLACE New manual pages
2000-12-24 14:19:00 +01:00
VOID(table->file->extra(HA_EXTRA_NO_IGNORE_DUP_KEY));
ha_innodb.cc: Remove accidenatlly committed debug printfs when query cache is used sql_update.cc, sql_load.cc, sql_insert.cc, sql_delete.cc: For the transactional query cache algorithm to work we must invalidate the query cache in INSERT/DELETE/UPDATE before writing to the binlog or calling ha_autocommit_... Note that binlog writing may also call commit. The crucial thing is that the transaction which modified data must not be committed BEFORE the query cache is invalidated.
2003-05-26 18:10:43 +02:00
/*
Invalidate the table in the query cache if something changed.
This must be before binlog writing and ha_autocommit_...
*/
if (updated)
Update for VC++ (Fixed project files, compiler warnings etc..)
2003-12-21 01:07:45 +01:00
{
ha_innodb.cc: Remove accidenatlly committed debug printfs when query cache is used sql_update.cc, sql_load.cc, sql_insert.cc, sql_delete.cc: For the transactional query cache algorithm to work we must invalidate the query cache in INSERT/DELETE/UPDATE before writing to the binlog or calling ha_autocommit_... Note that binlog writing may also call commit. The crucial thing is that the transaction which modified data must not be committed BEFORE the query cache is invalidated.
2003-05-26 18:10:43 +02:00
query_cache_invalidate3(thd, table_list, 1);
Update for VC++ (Fixed project files, compiler warnings etc..)
2003-12-21 01:07:45 +01:00
}
ha_innodb.cc: Remove accidenatlly committed debug printfs when query cache is used sql_update.cc, sql_load.cc, sql_insert.cc, sql_delete.cc: For the transactional query cache algorithm to work we must invalidate the query cache in INSERT/DELETE/UPDATE before writing to the binlog or calling ha_autocommit_... Note that binlog writing may also call commit. The crucial thing is that the transaction which modified data must not be committed BEFORE the query cache is invalidated.
2003-05-26 18:10:43 +02:00
Fix fur BUG#13348: "multi-table updates and deletes are not logged if no rows were affected". Not fixed in 4.1 as not critical. Also I'm correcting error checking of multi-UPDATE/DELETE when it comes to binlogging, to make it consistent with when we rollback the statement.
2005-09-20 17:41:47 +02:00
/*
error < 0 means really no error at all: we processed all rows until the
last one without error. error > 0 means an error (e.g. unique key
violation and no IGNORE or REPLACE). error == 0 is also an error (if
preparing the record or invoking before triggers fails). See
ha_autocommit_or_rollback(error>=0) and DBUG_RETURN(error>=0) below.
Sometimes we want to binlog even if we updated no rows, in case user used
it to be sure master and slave are in same state.
*/
if ((error < 0) || (updated && !transactional_table))
Import changeset
2000-07-31 21:29:14 +02:00
{
Faster log::write() call, memory leak fix, flush master fix.
2000-09-16 03:27:21 +02:00
if (mysql_bin_log.is_open())
{
Fix fur BUG#13348: "multi-table updates and deletes are not logged if no rows were affected". Not fixed in 4.1 as not critical. Also I'm correcting error checking of multi-UPDATE/DELETE when it comes to binlogging, to make it consistent with when we rollback the statement.
2005-09-20 17:41:47 +02:00
if (error < 0)
Fix for BUG#2083 "EE_ error codes (EE_DELETE, EE_WRITE) end up in the binlog, making slave stop". The problem was that during execution of the command on the master, an error can occur (for example, not space left on device, then mysqld waits and when there is space it completes successfully: so finally it worked but the error EE_WRITE remains in thd->net.last_errno and thd->net.last_error). To know if finally the command succeeded, we test the 'error' variable in every place, and if it shows no failure we reset thd->net.last_err* using the function THD::clear_error() which is backported from 4.1. A new test to see if now only real errors get to the binlog (note: the test uses "rm"). Also a bit of memory free/alloc saving in log_event.cc (do not free the whole mem_root after every query in the slave SQL thread: we can keep the initial block of it; which will be freed when the thread terminates).
2003-12-16 11:10:50 +01:00
thd->clear_error();
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
if (thd->binlog_query(THD::ROW_QUERY_TYPE,
thd->query, thd->query_length,
transactional_table, FALSE) &&
transactional_table)
{
Put temporary files in binlog cache when using BEGIN/COMMIT Let MySQL 4.0 read 4.1 .frm files without 4.1 specific extensions New variables @@rand_seed1 and @@rand_seed2 (used by replication) DROP TEMPORARY TABLE
2002-11-07 03:02:37 +01:00
error=1; // Rollback update
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
}
Faster log::write() call, memory leak fix, flush master fix.
2000-09-16 03:27:21 +02:00
}
query_id and my_xid -> ulonglong fix for binlog+autocommit+tclog comments, style fixes
2005-01-27 22:38:56 +01:00
if (!transactional_table)
All statements are now run in a sub transaction Give warning if we are using non transactional tables and do ROLLBACK Porting of crash-me and benchmarks to FrontBase
2000-11-24 00:51:18 +01:00
thd->options|=OPTION_STATUS_NO_TRANS_UPDATE;
Import changeset
2000-07-31 21:29:14 +02:00
}
Merge mysql.com:/home/jimw/my/mysql-4.1-clean into mysql.com:/home/jimw/my/mysql-5.0-clean
2005-12-12 19:11:56 +01:00
free_underlaid_joins(thd, select_lex);
Small improvement to alloc_root Add support for LIMIT # OFFSET # Changed lock handling: Now all locks should be stored in TABLE_LIST instead of passed to functions. Don't call query_cache_invalidate() twice in some cases mysql_change_user() now clears states to be equal to close + connect. Fixed a bug with multi-table-update and multi-table-delete when used with LOCK TABLES Fixed a bug with replicate-do and UPDATE
2002-11-16 19:19:10 +01:00
if (transactional_table)
{
if (ha_autocommit_or_rollback(thd, error >= 0))
error=1;
}
fixed invalidation of query cache excluded double call of 'invalidate()'
2002-11-21 23:33:15 +01:00
ixed wrong unlock of tables (new bug)
2003-05-26 19:48:40 +02:00
if (thd->lock)
{
mysql_unlock_tables(thd, thd->lock);
thd->lock=0;
}
WL#3146 "less locking in auto_increment": this is a cleanup patch for our current auto_increment handling: new names for auto_increment variables in THD, new methods to manipulate them (see sql_class.h), some move into handler::, causing less backup/restore work when executing substatements. This makes the logic hopefully clearer, less work is is needed in mysql_insert(). By cleaning up, using different variables for different purposes (instead of one for 3 things...), we fix those bugs, which someone may want to fix in 5.0 too: BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate statement-based" BUG#20341 "stored function inserting into one auto_increment puts bad data in slave" BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE" (now if a row is updated, LAST_INSERT_ID() will return its id) and re-fixes: BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT" (already fixed differently by Ramil in 4.1) Test of documented behaviour of mysql_insert_id() (there was no test). The behaviour changes introduced are: - LAST_INSERT_ID() now returns "the first autogenerated auto_increment value successfully inserted", instead of "the first autogenerated auto_increment value if any row was successfully inserted", see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY UPDATE, see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() does not change if no autogenerated value was successfully inserted (it used to then be 0), see auto_increment.test. - if in INSERT SELECT no autogenerated value was successfully inserted, mysql_insert_id() now returns the id of the last inserted row (it already did this for INSERT VALUES), see mysql_client_test.c. - if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X (it already did this for INSERT VALUES), see mysql_client_test.c. - NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE, the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID influences not only the first row now. Additionally, when unlocking a table we check that the thread is not keeping a next_insert_id (as the table is unlocked that id is potentially out-of-date); forgetting about this next_insert_id is done in a new handler::ha_release_auto_increment(). Finally we prepare for engines capable of reserving finite-length intervals of auto_increment values: we store such intervals in THD. The next step (to be done by the replication team in 5.1) is to read those intervals from THD and actually store them in the statement-based binary log. NDB will be a good engine to test that.
2006-07-09 17:52:19 +02:00
/* If LAST_INSERT_ID(X) was used, report X */
id= thd->arg_of_last_insert_id_function ?
thd->first_successful_insert_id_in_prev_stmt : 0;
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
if (error < 0)
Import changeset
2000-07-31 21:29:14 +02:00
{
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
Fix for the bug #841: wrong number of warnings
2003-07-14 12:32:31 +02:00
sprintf(buff, ER(ER_UPDATE_INFO), (ulong) found, (ulong) updated,
(ulong) thd->cuted_fields);
WL#1567: Add ROW_COUNT() Function to retrieve count of rows updated/inserted/deleted
2004-05-04 13:45:20 +02:00
thd->row_count_func=
(thd->client_capabilities & CLIENT_FOUND_ROWS) ? found : updated;
WL#3146 "less locking in auto_increment": this is a cleanup patch for our current auto_increment handling: new names for auto_increment variables in THD, new methods to manipulate them (see sql_class.h), some move into handler::, causing less backup/restore work when executing substatements. This makes the logic hopefully clearer, less work is is needed in mysql_insert(). By cleaning up, using different variables for different purposes (instead of one for 3 things...), we fix those bugs, which someone may want to fix in 5.0 too: BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate statement-based" BUG#20341 "stored function inserting into one auto_increment puts bad data in slave" BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE" (now if a row is updated, LAST_INSERT_ID() will return its id) and re-fixes: BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT" (already fixed differently by Ramil in 4.1) Test of documented behaviour of mysql_insert_id() (there was no test). The behaviour changes introduced are: - LAST_INSERT_ID() now returns "the first autogenerated auto_increment value successfully inserted", instead of "the first autogenerated auto_increment value if any row was successfully inserted", see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY UPDATE, see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() does not change if no autogenerated value was successfully inserted (it used to then be 0), see auto_increment.test. - if in INSERT SELECT no autogenerated value was successfully inserted, mysql_insert_id() now returns the id of the last inserted row (it already did this for INSERT VALUES), see mysql_client_test.c. - if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X (it already did this for INSERT VALUES), see mysql_client_test.c. - NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE, the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID influences not only the first row now. Additionally, when unlocking a table we check that the thread is not keeping a next_insert_id (as the table is unlocked that id is potentially out-of-date); forgetting about this next_insert_id is done in a new handler::ha_release_auto_increment(). Finally we prepare for engines capable of reserving finite-length intervals of auto_increment values: we store such intervals in THD. The next step (to be done by the replication team in 5.1) is to read those intervals from THD and actually store them in the statement-based binary log. NDB will be a good engine to test that.
2006-07-09 17:52:19 +02:00
send_ok(thd, (ulong) thd->row_count_func, id, buff);
Remove compiler warnings (Mostly in DBUG_PRINT() and unused arguments) Fixed bug in query cache when used with traceing (--with-debug) Fixed memory leak in mysqldump Removed warnings from mysqltest scripts (replaced -- with #)
2006-11-20 21:42:06 +01:00
DBUG_PRINT("info",("%ld records updated", (long) updated));
Import changeset
2000-07-31 21:29:14 +02:00
}
Add warnings for single line inserts. To do this I had to convert count_cuted_fields to an enum (to be able to detect setting a NOT NULL field to NULL)
2003-10-11 22:26:39 +02:00
thd->count_cuted_fields= CHECK_FIELD_IGNORE; /* calc cuted fields */
Fixed bug in strict mode (A state was not properly cleared on successful insert/update's)
2004-10-05 00:05:15 +02:00
thd->abort_on_warning= 0;
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN((error >= 0 || thd->net.report_error) ? 1 : 0);
Fix problem where key_read was not cleaned up properly, which caused assert in innodb test.
2003-05-03 15:21:43 +02:00
err:
delete select;
Few more places with thd->lex replaced with lex (mysql_update, check_simple_select). This change was inspired by Monty in December 2003 when thd->lex became a pointer in 4.1.
2004-11-11 20:37:48 +01:00
free_underlaid_joins(thd, select_lex);
Fix problem where key_read was not cleaned up properly, which caused assert in innodb test.
2003-05-03 15:21:43 +02:00
if (table->key_read)
{
table->key_read=0;
table->file->extra(HA_EXTRA_NO_KEYREAD);
}
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
thd->abort_on_warning= 0;
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(1);
Import changeset
2000-07-31 21:29:14 +02:00
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
after review PS fixes
2004-04-10 00:14:32 +02:00
/*
Prepare items in UPDATE statement
SYNOPSIS
mysql_prepare_update()
thd - thread handler
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
table_list - global/local table list
after review PS fixes
2004-04-10 00:14:32 +02:00
conds - conditions
order_num - number of ORDER BY list entries
order - ORDER BY clause list
RETURN VALUE
post-review fixes
2004-11-12 13:34:00 +01:00
FALSE OK
TRUE error
after review PS fixes
2004-04-10 00:14:32 +02:00
*/
post-review fixes
2004-11-12 13:34:00 +01:00
bool mysql_prepare_update(THD *thd, TABLE_LIST *table_list,
after review PS fixes
2004-04-10 00:14:32 +02:00
Item **conds, uint order_num, ORDER *order)
{
Fixed bug #21493: crash for the second execution of a function containing a select statement that uses an aggregating IN subquery. Added a parameter to the function fix_prepare_information to restore correctly the having clause for the second execution. Saved andor structure of the having conditions at the proper moment before any calls of split_sum_func2 that could modify the having structure adding new Item_ref objects. (These additions, are produced not with the statement mem_root, but rather with the execution mem_root.)
2006-09-16 18:50:48 +02:00
Item *fake_conds= 0;
after review PS fixes
2004-04-10 00:14:32 +02:00
TABLE *table= table_list->table;
TABLE_LIST tables;
List<Item> all_fields;
PS and SP made compatible in mechanism used for preparing query for rexecutions (Bug #2266)
2004-05-20 01:02:49 +02:00
SELECT_LEX *select_lex= &thd->lex->select_lex;
after review PS fixes
2004-04-10 00:14:32 +02:00
DBUG_ENTER("mysql_prepare_update");
#ifndef NO_EMBEDDED_ACCESS_CHECKS
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
table_list->grant.want_privilege= table->grant.want_privilege=
(SELECT_ACL & ~table->grant.privilege);
support of view underlying tables and SP functions security check added (BUG#9505) (WL#2787)
2005-10-27 23:18:23 +02:00
table_list->register_want_access(SELECT_ACL);
after review PS fixes
2004-04-10 00:14:32 +02:00
#endif
bzero((char*) &tables,sizeof(tables)); // For ORDER BY
tables.table= table;
tables.alias= table_list->alias;
Fixed bug #12762: allowed set functions aggregated in outer subqueries, allowed nested set functions.
2005-10-15 23:32:37 +02:00
thd->lex->allow_sum_func= 0;
after review PS fixes
2004-04-10 00:14:32 +02:00
BUG#18681: View privileges are broken The check for view security was lacking several points : 1. Check with the right set of permissions : for each table ref that participates in a view there were the right credentials to use in it's security_ctx member, but these weren't used for checking the credentials. This makes hard enforcing the SQL SECURITY DEFINER|INVOKER property consistently. 2. Because of the above the security checking for views was just ruled out in explicit ways in several places. 3. The security was checked only for the columns of the tables that are brought into the query from a view. So if there is no column reference outside of the view definition it was not detecting the lack of access to the tables in the view in SQL SECURITY INVOKER mode. The fix below tries to fix the above 3 points.
2006-05-26 10:47:53 +02:00
if (setup_tables_and_check_access(thd, &select_lex->context,
&select_lex->top_join_list,
Merge bk-internal.mysql.com:/home/bk/mysql-5.1-new into mysql.com:/home/my/mysql-5.1
2006-06-04 20:05:22 +02:00
table_list,
BUG#18681: View privileges are broken The check for view security was lacking several points : 1. Check with the right set of permissions : for each table ref that participates in a view there were the right credentials to use in it's security_ctx member, but these weren't used for checking the credentials. This makes hard enforcing the SQL SECURITY DEFINER|INVOKER property consistently. 2. Because of the above the security checking for views was just ruled out in explicit ways in several places. 3. The security was checked only for the columns of the tables that are brought into the query from a view. So if there is no column reference outside of the view definition it was not detecting the lack of access to the tables in the view in SQL SECURITY INVOKER mode. The fix below tries to fix the above 3 points.
2006-05-26 10:47:53 +02:00
&select_lex->leaf_tables,
Fixed bug#21261: Wrong access rights was required for an insert into a view SELECT right instead of INSERT right was required for an insert into to a view. This wrong behaviour appeared after the fix for bug #20989. Its intention was to ask only SELECT right for all tables except the very first for a complex INSERT query. But that patch has done it in a wrong way and lead to asking a wrong access right for an insert into a view. The setup_tables_and_check_access() function now accepts two want_access parameters. One will be used for the first table and the second for other tables.
2006-08-15 19:45:24 +02:00
FALSE, UPDATE_ACL, SELECT_ACL) ||
fixed merged view fields names (BUG#5147) support of merged VIEW over several tables added (WL#1809)
2004-09-14 18:28:29 +02:00
setup_conds(thd, table_list, select_lex->leaf_tables, conds) ||
PS and SP made compatible in mechanism used for preparing query for rexecutions (Bug #2266)
2004-05-20 01:02:49 +02:00
select_lex->setup_ref_array(thd, order_num) ||
setup_order(thd, select_lex->ref_pointer_array,
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
table_list, all_fields, all_fields, order) ||
PS and SP made compatible in mechanism used for preparing query for rexecutions (Bug #2266)
2004-05-20 01:02:49 +02:00
setup_ftfuncs(select_lex))
post-review fixes
2004-11-12 13:34:00 +01:00
DBUG_RETURN(TRUE);
after review PS fixes
2004-04-10 00:14:32 +02:00
/* Check that we are not using table that we are updating in a sub select */
{
issue correct error message in case of view presence for duplicated table on update (BUG#10773) frequently used command sequence replaced with inline function
2005-08-02 21:54:49 +02:00
TABLE_LIST *duplicate;
BUG#5390 - problems with merge tables Problem #1: INSERT...SELECT, Version for 5.1. Extended the unique table check by a check of lock data. Merge sub-tables cannot be detected by doing name checks only.
2005-12-22 13:48:00 +01:00
if ((duplicate= unique_table(thd, table_list, table_list->next_global)))
issue correct error message in case of view presence for duplicated table on update (BUG#10773) frequently used command sequence replaced with inline function
2005-08-02 21:54:49 +02:00
{
update_non_unique_table_error(table_list, "UPDATE", duplicate);
my_error(ER_UPDATE_TABLE_USED, MYF(0), table_list->table_name);
DBUG_RETURN(TRUE);
}
after review PS fixes
2004-04-10 00:14:32 +02:00
}
Fixed bug #21493: crash for the second execution of a function containing a select statement that uses an aggregating IN subquery. Added a parameter to the function fix_prepare_information to restore correctly the having clause for the second execution. Saved andor structure of the having conditions at the proper moment before any calls of split_sum_func2 that could modify the having structure adding new Item_ref objects. (These additions, are produced not with the statement mem_root, but rather with the execution mem_root.)
2006-09-16 18:50:48 +02:00
select_lex->fix_prepare_information(thd, conds, &fake_conds);
post-review fixes
2004-11-12 13:34:00 +01:00
DBUG_RETURN(FALSE);
after review PS fixes
2004-04-10 00:14:32 +02:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
/***************************************************************************
Big code cleanup/review before 4.0.2 release. (All commit emails since 4.0.1 checked) This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
2002-06-11 10:20:31 +02:00
Update multiple tables from join
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
***************************************************************************/
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/*
Reverted patch for new usage of open_count as it caused more problems than it solved Cleaned up patch for checking locks for multi-table updates
2004-10-06 00:24:21 +02:00
Get table map for list of Item_field
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
*/
Reverted patch for new usage of open_count as it caused more problems than it solved Cleaned up patch for checking locks for multi-table updates
2004-10-06 00:24:21 +02:00
static table_map get_table_map(List<Item> *items)
{
List_iterator_fast<Item> item_it(*items);
Item_field *item;
table_map map= 0;
while ((item= (Item_field *) item_it++))
map|= item->used_tables();
Remove compiler warnings (Mostly in DBUG_PRINT() and unused arguments) Fixed bug in query cache when used with traceing (--with-debug) Fixed memory leak in mysqldump Removed warnings from mysqltest scripts (replaced -- with #)
2006-11-20 21:42:06 +01:00
DBUG_PRINT("info", ("table_map: 0x%08lx", (long) map));
Reverted patch for new usage of open_count as it caused more problems than it solved Cleaned up patch for checking locks for multi-table updates
2004-10-06 00:24:21 +02:00
return map;
}
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
/*
make update specific preparation and checks after opening tables
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
SYNOPSIS
mysql_multi_update_prepare()
thd thread handler
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
RETURN
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
FALSE OK
TRUE Error
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
*/
allow UPDATE and DELETE stetements with tables derived from subquery if they are not updated (BUG#2117) allow delete table by alias in multi-delete statement
2004-01-04 22:44:33 +01:00
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
bool mysql_multi_update_prepare(THD *thd)
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
{
LEX *lex= thd->lex;
TABLE_LIST *table_list= lex->query_tables;
After merge fixes
2004-12-30 23:44:00 +01:00
TABLE_LIST *tl, *leaves;
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
List<Item> *fields= &lex->select_lex.item_list;
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
table_map tables_for_update;
fixed using VIEW fields (BUG#4617)
2004-07-23 08:20:58 +02:00
bool update_view= 0;
post merge
2004-11-21 18:33:49 +01:00
/*
if this multi-update was converted from usual update, here is table
counter else junk will be assigned here, but then replaced with real
count in open_tables()
*/
uint table_count= lex->table_count;
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
const bool using_lock_tables= thd->locked_tables != 0;
post merge
2004-11-21 18:33:49 +01:00
bool original_multiupdate= (thd->lex->sql_command == SQLCOM_UPDATE_MULTI);
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
bool need_reopen= FALSE;
After merge fixes
2004-12-30 23:44:00 +01:00
DBUG_ENTER("mysql_multi_update_prepare");
post-merge fix
2004-11-25 01:23:13 +01:00
/* following need for prepared statements, to run next time multi-update */
thd->lex->sql_command= SQLCOM_UPDATE_MULTI;
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
reopen_tables:
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
/* open tables and create derived ones, but do not lock and fill them */
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
if (((original_multiupdate || need_reopen) &&
Fix for bug #10055 "Using stored function with information_schema causes empty result set". To enable full access to contents of I_S tables from stored functions or statements that use them, we manipulate with thread's open tables state and ensure that we won't cause deadlock when we open tables by ignoring flushes and name-locks. Building of contents of I_S.TABLES no longer requires locking of tables since we use use handler::info() method with HA_STATUS_AUTO flag instead of handler::update_auto_increment() for obtaining information about auto-increment values. But this also means that handlers have to implement support for HA_STATUS_AUTO flag (particularly InnoDB needs it).
2005-08-08 15:46:06 +02:00
open_tables(thd, &table_list, &table_count, 0)) ||
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
mysql_handle_derived(lex, &mysql_derived_prepare))
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
DBUG_RETURN(TRUE);
post merge
2004-11-21 18:33:49 +01:00
/*
setup_tables() need for VIEWs. JOIN::prepare() will call setup_tables()
second time, but this call will do nothing (there are check for second
call in setup_tables()).
*/
merge
2004-11-21 19:08:12 +01:00
BUG#18681: View privileges are broken The check for view security was lacking several points : 1. Check with the right set of permissions : for each table ref that participates in a view there were the right credentials to use in it's security_ctx member, but these weren't used for checking the credentials. This makes hard enforcing the SQL SECURITY DEFINER|INVOKER property consistently. 2. Because of the above the security checking for views was just ruled out in explicit ways in several places. 3. The security was checked only for the columns of the tables that are brought into the query from a view. So if there is no column reference outside of the view definition it was not detecting the lack of access to the tables in the view in SQL SECURITY INVOKER mode. The fix below tries to fix the above 3 points.
2006-05-26 10:47:53 +02:00
if (setup_tables_and_check_access(thd, &lex->select_lex.context,
&lex->select_lex.top_join_list,
Merge bk-internal.mysql.com:/home/bk/mysql-5.1-new into mysql.com:/home/my/mysql-5.1
2006-06-04 20:05:22 +02:00
table_list,
BUG#18681: View privileges are broken The check for view security was lacking several points : 1. Check with the right set of permissions : for each table ref that participates in a view there were the right credentials to use in it's security_ctx member, but these weren't used for checking the credentials. This makes hard enforcing the SQL SECURITY DEFINER|INVOKER property consistently. 2. Because of the above the security checking for views was just ruled out in explicit ways in several places. 3. The security was checked only for the columns of the tables that are brought into the query from a view. So if there is no column reference outside of the view definition it was not detecting the lack of access to the tables in the view in SQL SECURITY INVOKER mode. The fix below tries to fix the above 3 points.
2006-05-26 10:47:53 +02:00
&lex->select_lex.leaf_tables, FALSE,
Fixed bug#21261: Wrong access rights was required for an insert into a view SELECT right instead of INSERT right was required for an insert into to a view. This wrong behaviour appeared after the fix for bug #20989. Its intention was to ask only SELECT right for all tables except the very first for a complex INSERT query. But that patch has done it in a wrong way and lead to asking a wrong access right for an insert into a view. The setup_tables_and_check_access() function now accepts two want_access parameters. One will be used for the first table and the second for other tables.
2006-08-15 19:45:24 +02:00
UPDATE_ACL, SELECT_ACL))
merge
2004-11-21 19:08:12 +01:00
DBUG_RETURN(TRUE);
Ensure that privileges are tested properly for multi-table-updates. Now one need only SELECT privilege for tables that are only read in UPDATE statements with many tables. (Bug #2377).
2004-02-03 19:17:23 +01:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (setup_fields_with_no_wrap(thd, 0, *fields, MARK_COLUMNS_WRITE, 0, 0))
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
DBUG_RETURN(TRUE);
fixed using VIEW fields (BUG#4617)
2004-07-23 08:20:58 +02:00
for (tl= table_list; tl ; tl= tl->next_local)
{
if (tl->view)
{
update_view= 1;
break;
}
}
if (update_view && check_fields(thd, *fields))
allow UPDATE and DELETE stetements with tables derived from subquery if they are not updated (BUG#2117) allow delete table by alias in multi-delete statement
2004-01-04 22:44:33 +01:00
{
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
DBUG_RETURN(TRUE);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
}
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
tables_for_update= get_table_map(fields);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/*
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
Setup timestamp handling and locking mode
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
*/
Name resolution context added (BUG#6443)
2005-07-01 06:05:42 +02:00
leaves= lex->select_lex.leaf_tables;
fixed merged view fields names (BUG#5147) support of merged VIEW over several tables added (WL#1809)
2004-09-14 18:28:29 +02:00
for (tl= leaves; tl; tl= tl->next_leaf)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE *table= tl->table;
post merge
2004-11-21 18:33:49 +01:00
/* Only set timestamp column if this is not modified */
WL#1266 "Separate auto-set logic from TIMESTAMP type." Final version of patch. Adds support for specifying of DEFAULT NOW() and/or ON UPDATE NOW() clauses for TIMESTAMP field definition. Current implementation allows only one such field per table and uses several unireg types for storing info about this properties of field. It should be replaced with better implementation when new .frm format is introduced.
2004-04-02 08:12:53 +02:00
if (table->timestamp_field &&
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
bitmap_is_set(table->write_set,
table->timestamp_field->field_index))
Fixes after merge with 4.1 FOUND is not a reserved keyword anymore Added Item_field::set_no_const_sub() to be able to mark fields that can't be substituted Added 'simple_select' method to be able to quickly determinate if a select_result is a normal SELECT Note that the 5.0 tree is not yet up to date: Sanja will have to fix multi-update-locks for this merge to be complete
2004-11-03 11:39:38 +01:00
table->timestamp_field_type= TIMESTAMP_NO_AUTO_SET;
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
/* if table will be updated then check that it is unique */
if (table->map & tables_for_update)
allow UPDATE and DELETE stetements with tables derived from subquery if they are not updated (BUG#2117) allow delete table by alias in multi-delete statement
2004-01-04 22:44:33 +01:00
{
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
if (!tl->updatable || check_key_in_view(thd, tl))
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
{
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
my_error(ER_NON_UPDATABLE_TABLE, MYF(0), tl->alias, "UPDATE");
merge
2004-11-12 14:36:31 +01:00
DBUG_RETURN(TRUE);
Added --compact to mysqlbinlog Fixed output from mysqlbinlog when using --skip-comments Fixed warnings from valgrind Fixed ref_length when used with HEAP tables More efficent need_conversion() Fixed error handling in UPDATE with not updateable tables Fixed bug in null handling in CAST to signed/unsigned
2004-02-09 12:31:03 +01:00
}
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
table->mark_columns_needed_for_update();
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
DBUG_PRINT("info",("setting table `%s` for update", tl->alias));
Fix for bug #9486 "Can't perform multi-update in stored procedure". New more SP-locking friendly approach to handling locks in multi-update. Now we mark all tables of multi-update as needing write lock at parsing stage and if possible downgrade lock at execution stage (For its work SP-locking mechanism needs to know all lock types right after parsing stage).
2005-04-15 18:31:47 +02:00
/*
If table will be updated we should not downgrade lock for it and
leave it as is.
*/
After merge fixes
2004-12-30 23:44:00 +01:00
}
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
else
{
DBUG_PRINT("info",("setting table `%s` for read-only", tl->alias));
Merge with 4.1
2005-02-22 14:47:00 +01:00
/*
If we are using the binary log, we need TL_READ_NO_INSERT to get
correct order of statements. Otherwise, we use a TL_READ lock to
improve performance.
*/
tl->lock_type= using_update_log ? TL_READ_NO_INSERT : TL_READ;
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
tl->updating= 0;
Fix for bug #9486 "Can't perform multi-update in stored procedure". New more SP-locking friendly approach to handling locks in multi-update. Now we mark all tables of multi-update as needing write lock at parsing stage and if possible downgrade lock at execution stage (For its work SP-locking mechanism needs to know all lock types right after parsing stage).
2005-04-15 18:31:47 +02:00
/* Update TABLE::lock_type accordingly. */
if (!tl->placeholder() && !tl->schema_table && !using_lock_tables)
tl->table->reginfo.lock_type= tl->lock_type;
allow UPDATE and DELETE stetements with tables derived from subquery if they are not updated (BUG#2117) allow delete table by alias in multi-delete statement
2004-01-04 22:44:33 +01:00
}
Many files: Fixed bug #8528. Representation for single-table views was made similar to representation for multi-table views. view.test: Added test case for bug #8528. view.result: Added test case for bug #8528. Fixed other test cases.
2005-05-11 01:31:13 +02:00
}
Code cleanups during code reviews Ensure we get error if INSERT IGNORE ... SELECT fails Fixed wrong key_part->key_length usage in index_merge
2005-06-01 15:35:09 +02:00
for (tl= table_list; tl; tl= tl->next_local)
Many files: Fixed bug #8528. Representation for single-table views was made similar to representation for multi-table views. view.test: Added test case for bug #8528. view.result: Added test case for bug #8528. Fixed other test cases.
2005-05-11 01:31:13 +02:00
{
Merge with 4.1
2004-12-22 12:54:39 +01:00
/* Check access privileges for table */
Many files: Fixed bug #8528. Representation for single-table views was made similar to representation for multi-table views. view.test: Added test case for bug #8528. view.result: Added test case for bug #8528. Fixed other test cases.
2005-05-11 01:31:13 +02:00
if (!tl->derived)
allow UPDATE and DELETE stetements with tables derived from subquery if they are not updated (BUG#2117) allow delete table by alias in multi-delete statement
2004-01-04 22:44:33 +01:00
{
After merge fixes
2004-12-30 23:44:00 +01:00
uint want_privilege= tl->updating ? UPDATE_ACL : SELECT_ACL;
if (check_access(thd, want_privilege,
Bug#9683 INFORMATION_SCH: Creation of temporary table allowed in Information_schema DB Bug#9846 Inappropriate error displayed while dropping table from 'INFORMATION_SCHEMA' Bug#10734 Grant of privileges other than 'select' and 'create view' should fail on schema Bug#10708 SP's can use INFORMATION_SCHEMA as ROUTINE_SCHEMA cumulative fix for bugs above(after review, 2nd version) added privilege check for information schema db & tables
2005-09-13 13:07:38 +02:00
tl->db, &tl->grant.privilege, 0, 0,
test(tl->schema_table)) ||
After merge fixes
2004-12-30 23:44:00 +01:00
(grant_option && check_grant(thd, want_privilege, tl, 0, 1, 0)))
Merge with 4.1
2004-12-22 12:54:39 +01:00
DBUG_RETURN(TRUE);
allow UPDATE and DELETE stetements with tables derived from subquery if they are not updated (BUG#2117) allow delete table by alias in multi-delete statement
2004-01-04 22:44:33 +01:00
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
postreview fixes
2004-11-08 00:54:23 +01:00
post merge
2004-11-21 18:33:49 +01:00
/* check single table update for view compound from several tables */
for (tl= table_list; tl; tl= tl->next_local)
{
Many files: Fixed bug #8528. Representation for single-table views was made similar to representation for multi-table views. view.test: Added test case for bug #8528. view.result: Added test case for bug #8528. Fixed other test cases.
2005-05-11 01:31:13 +02:00
if (tl->effective_algorithm == VIEW_ALGORITHM_MERGE)
post merge
2004-11-21 18:33:49 +01:00
{
TABLE_LIST *for_update= 0;
Many files: Fixed bug #8528. Representation for single-table views was made similar to representation for multi-table views. view.test: Added test case for bug #8528. view.result: Added test case for bug #8528. Fixed other test cases.
2005-05-11 01:31:13 +02:00
if (tl->check_single_table(&for_update, tables_for_update, tl))
post merge
2004-11-21 18:33:49 +01:00
{
my_error(ER_VIEW_MULTIUPDATE, MYF(0),
tl->view_db.str, tl->view_name.str);
DBUG_RETURN(-1);
}
}
}
new lock for multiupdate: - open and create derived tables - detect which tables should be locked for write - lock and fill derived tables some unitialized variables fixed
2004-11-05 16:29:47 +01:00
/* now lock and fill tables */
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
if (lock_tables(thd, table_list, table_count, &need_reopen))
postreview fixes
2004-11-08 00:54:23 +01:00
{
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
if (!need_reopen)
DBUG_RETURN(TRUE);
postreview fixes
2004-11-08 00:54:23 +01:00
/*
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
We have to reopen tables since some of them were altered or dropped
during lock_tables() or something was done with their triggers.
Let us do some cleanups to be able do setup_table() and setup_fields()
once again.
postreview fixes
2004-11-08 00:54:23 +01:00
*/
List_iterator_fast<Item> it(*fields);
Item *item;
After merge fixes Fixed compiler warnings Fix core dump when sending SIGHUP to mysqld
2004-12-06 16:15:54 +01:00
while ((item= it++))
postreview fixes
2004-11-08 00:54:23 +01:00
item->cleanup();
After merge fixes Fixed compiler warnings Fix core dump when sending SIGHUP to mysqld
2004-12-06 16:15:54 +01:00
/* We have to cleanup translation tables of views. */
postreview fixes
2004-11-08 00:54:23 +01:00
for (TABLE_LIST *tbl= table_list; tbl; tbl= tbl->next_global)
tbl->cleanup_items();
Fix for bug #16593 "Deadlock or crash in stress test for case where trigger starts trigger". In short, the deadlock/crash happened when execution of statement, which used stored functions or activated triggers, coincided with alteration of the tables used by these functions or triggers (in highly concurrent environment). Bug was caused by the incorrect handling of tables from prelocked set in open_tables() functions in situations when refresh happened. This fix replaces old smart but not very robust way of handling tables after refresh (which was closing only old tables), with new one which simply closes all tables opened so far and restarts open_tables(). Also fixed handling of temporary tables in close_tables_for_reopen(). No test case present since bug manifests itself only in concurrent environment.
2006-02-16 14:19:24 +01:00
close_tables_for_reopen(thd, &table_list);
Fix for bug #12704 "Server crashes during trigger execution". This bug occurs when some trigger for table used by DML statement is created or changed while statement was waiting in lock_tables(). In this situation prelocking set which we have calculated becames invalid which can easily lead to errors and even in some cases to crashes. With proposed patch we no longer silently reopen tables in lock_tables(), instead caller of lock_tables() becomes responsible for reopening tables and recalculation of prelocking set.
2005-09-15 01:56:09 +02:00
goto reopen_tables;
postreview fixes
2004-11-08 00:54:23 +01:00
}
After merge fixes
2004-12-30 23:44:00 +01:00
fixed mechanism of detection selection from table wich we update (BUG##9398, BUG#8703) fixed wrong join view detection in multi-delete which lead to server crash
2005-03-28 14:13:31 +02:00
/*
Check that we are not using table that we are updating, but we should
skip all tables of UPDATE SELECT itself
*/
lex->select_lex.exclude_from_table_unique_test= TRUE;
After merge fixes
2004-12-30 23:44:00 +01:00
/* We only need SELECT privilege for columns in the values list */
for (tl= leaves; tl; tl= tl->next_leaf)
{
TABLE *table= tl->table;
TABLE_LIST *tlist;
issue correct error message in case of view presence for duplicated table on update (BUG#10773) frequently used command sequence replaced with inline function
2005-08-02 21:54:49 +02:00
if (!(tlist= tl->top_table())->derived)
After merge fixes
2004-12-30 23:44:00 +01:00
{
tlist->grant.want_privilege=
(SELECT_ACL & ~tlist->grant.privilege);
table->grant.want_privilege= (SELECT_ACL & ~table->grant.privilege);
}
DBUG_PRINT("info", ("table: %s want_privilege: %u", tl->alias,
(uint) table->grant.want_privilege));
fixed mechanism of detection selection from table wich we update (BUG##9398, BUG#8703) fixed wrong join view detection in multi-delete which lead to server crash
2005-03-28 14:13:31 +02:00
if (tl->lock_type != TL_READ &&
issue correct error message in case of view presence for duplicated table on update (BUG#10773) frequently used command sequence replaced with inline function
2005-08-02 21:54:49 +02:00
tl->lock_type != TL_READ_NO_INSERT)
fixed mechanism of detection selection from table wich we update (BUG##9398, BUG#8703) fixed wrong join view detection in multi-delete which lead to server crash
2005-03-28 14:13:31 +02:00
{
issue correct error message in case of view presence for duplicated table on update (BUG#10773) frequently used command sequence replaced with inline function
2005-08-02 21:54:49 +02:00
TABLE_LIST *duplicate;
BUG#5390 - problems with merge tables Problem #1: INSERT...SELECT, Version for 5.1. Extended the unique table check by a check of lock data. Merge sub-tables cannot be detected by doing name checks only.
2005-12-22 13:48:00 +01:00
if ((duplicate= unique_table(thd, tl, table_list)))
issue correct error message in case of view presence for duplicated table on update (BUG#10773) frequently used command sequence replaced with inline function
2005-08-02 21:54:49 +02:00
{
update_non_unique_table_error(table_list, "UPDATE", duplicate);
DBUG_RETURN(TRUE);
}
fixed mechanism of detection selection from table wich we update (BUG##9398, BUG#8703) fixed wrong join view detection in multi-delete which lead to server crash
2005-03-28 14:13:31 +02:00
}
After merge fixes
2004-12-30 23:44:00 +01:00
}
Addition to fix for BUG#18036 - update of table joined to self reports table as crashed Set exclude_from_table_unique_test value back to FALSE. It is needed for further check in multi_update::prepare whether to use record cache.
2006-06-19 14:50:52 +02:00
/*
Set exclude_from_table_unique_test value back to FALSE. It is needed for
further check in multi_update::prepare whether to use record cache.
*/
lex->select_lex.exclude_from_table_unique_test= FALSE;
postreview fixes
2004-11-08 00:54:23 +01:00
if (thd->fill_derived_tables() &&
mysql_handle_derived(lex, &mysql_derived_filling))
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
DBUG_RETURN(TRUE);
fixed mechanism of detection selection from table wich we update (BUG##9398, BUG#8703) fixed wrong join view detection in multi-delete which lead to server crash
2005-03-28 14:13:31 +02:00
merge
2004-11-12 14:36:31 +01:00
DBUG_RETURN (FALSE);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
}
merge with 4.1
2004-10-29 18:26:52 +02:00
/*
Setup multi-update handling and call SELECT to do the join
*/
Bug#4118: multi-table UPDATE takes WRITE lock on read table Ensures that WRITE lock is not obtained on all tables referenced.
2004-10-03 01:20:47 +02:00
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
bool mysql_multi_update(THD *thd,
TABLE_LIST *table_list,
List<Item> *fields,
List<Item> *values,
COND *conds,
Fixes during review of new pushed code Removed duplicate usage of TMP_TABLE_FORCE_MYISAM by making 'options' longlong
2005-08-12 12:54:42 +02:00
ulonglong options,
Merge with 4.1 tree to get fix for INSERT IGNORE ... ON DUPLICATE KEY
2005-01-03 22:04:52 +01:00
enum enum_duplicates handle_duplicates, bool ignore,
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
SELECT_LEX_UNIT *unit, SELECT_LEX *select_lex)
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
{
multi_update *result;
DBUG_ENTER("mysql_multi_update");
Clumsy but working fix for bug #11973 "SELECT .. INTO var_name; in trigger cause crash on update". Let us update "thd" pointer in LEX, all its units and in LEX::result before executing statement in trigger body, since triggers are associated with TABLE object and because of this can be used in different threads.
2005-08-09 22:23:56 +02:00
if (!(result= new multi_update(table_list,
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
thd->lex->select_lex.leaf_tables,
fields, values,
Merge with 4.1 tree to get fix for INSERT IGNORE ... ON DUPLICATE KEY
2005-01-03 22:04:52 +01:00
handle_duplicates, ignore)))
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
DBUG_RETURN(TRUE);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
thd->no_trans_update= 0;
thd->abort_on_warning= test(thd->variables.sql_mode &
(MODE_STRICT_TRANS_TABLES |
MODE_STRICT_ALL_TABLES));
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
List<Item> total_list;
Remove compiler warnings and remove not used variables (Found during build process)
2005-02-25 15:53:22 +01:00
(void) mysql_select(thd, &select_lex->ref_pointer_array,
table_list, select_lex->with_wild,
total_list,
conds, 0, (ORDER *) NULL, (ORDER *)NULL, (Item *) NULL,
(ORDER *)NULL,
options | SELECT_NO_JOIN_CACHE | SELECT_NO_UNLOCK |
OPTION_SETUP_TABLES_DONE,
result, unit, select_lex);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
delete result;
Strict mode & better warnings Under strict mode MySQL will generate an error message if there was any conversion when assigning data to a field. Added checking of date/datetime fields. If strict mode, give error if we have not given value to field without a default value (for INSERT)
2004-09-28 19:08:00 +02:00
thd->abort_on_warning= 0;
Fix for bug #9486 "Can't perform multi-update in stored procedure". New more SP-locking friendly approach to handling locks in multi-update. Now we mark all tables of multi-update as needing write lock at parsing stage and if possible downgrade lock at execution stage (For its work SP-locking mechanism needs to know all lock types right after parsing stage).
2005-04-15 18:31:47 +02:00
DBUG_RETURN(FALSE);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Clumsy but working fix for bug #11973 "SELECT .. INTO var_name; in trigger cause crash on update". Let us update "thd" pointer in LEX, all its units and in LEX::result before executing statement in trigger body, since triggers are associated with TABLE object and because of this can be used in different threads.
2005-08-09 22:23:56 +02:00
multi_update::multi_update(TABLE_LIST *table_list,
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
TABLE_LIST *leaves_list,
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
List<Item> *field_list, List<Item> *value_list,
After merge fixes Add support for warnings for prepare of prepared statements Fixed test to work with --ps-protocol Fixed some test results
2005-01-04 12:46:53 +01:00
enum enum_duplicates handle_duplicates_arg,
bool ignore_arg)
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
:all_tables(table_list), leaves(leaves_list), update_tables(0),
Clumsy but working fix for bug #11973 "SELECT .. INTO var_name; in trigger cause crash on update". Let us update "thd" pointer in LEX, all its units and in LEX::result before executing statement in trigger body, since triggers are associated with TABLE object and because of this can be used in different threads.
2005-08-09 22:23:56 +02:00
tmp_tables(0), updated(0), found(0), fields(field_list),
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
values(value_list), table_count(0), copy_field(0),
handle_duplicates(handle_duplicates_arg), do_update(1), trans_safe(0),
Merge with 4.1 tree to get fix for INSERT IGNORE ... ON DUPLICATE KEY
2005-01-03 22:04:52 +01:00
transactional_tables(1), ignore(ignore_arg)
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
{}
/*
Connect fields with tables and create list of tables that are updated
*/
Added missing SSL library (Should be in source distribution) Fixed compiler warnings (a lot of hidden variables detected by the Forte compiler) Added a lot of 'version_xxx' strings to 'show variables' Prevent copying of TMP_TABLE_PARAM (This caused core dump bug on Solaris) Fixed problem with printing sub selects to debug log
2003-11-28 11:18:13 +01:00
int multi_update::prepare(List<Item> &not_used_values,
SELECT_LEX_UNIT *lex_unit)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE_LIST *table_ref;
SQL_LIST update;
Reverted patch for new usage of open_count as it caused more problems than it solved Cleaned up patch for checking locks for multi-table updates
2004-10-06 00:24:21 +02:00
table_map tables_to_update;
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Item_field *item;
List_iterator_fast<Item> field_it(*fields);
List_iterator_fast<Item> value_it(*values);
uint i, max_fields;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
DBUG_ENTER("multi_update::prepare");
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Add warnings for single line inserts. To do this I had to convert count_cuted_fields to an enum (to be able to detect setting a NOT NULL field to NULL)
2003-10-11 22:26:39 +02:00
thd->count_cuted_fields= CHECK_FIELD_WARN;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
thd->cuted_fields=0L;
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
thd->proc_info="updating main table";
Reverted patch for new usage of open_count as it caused more problems than it solved Cleaned up patch for checking locks for multi-table updates
2004-10-06 00:24:21 +02:00
tables_to_update= get_table_map(fields);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (!tables_to_update)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
post-review fixes
2004-11-12 13:34:00 +01:00
my_message(ER_NO_TABLES_USED, ER(ER_NO_TABLES_USED), MYF(0));
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(1);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Big code cleanup/review before 4.0.2 release. (All commit emails since 4.0.1 checked) This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
2002-06-11 10:20:31 +02:00
/*
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
We have to check values after setup_tables to get used_keys right in
reference tables
Big code cleanup/review before 4.0.2 release. (All commit emails since 4.0.1 checked) This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
2002-06-11 10:20:31 +02:00
*/
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (setup_fields(thd, 0, *values, MARK_COLUMNS_READ, 0, 0))
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
DBUG_RETURN(1);
Big code cleanup/review before 4.0.2 release. (All commit emails since 4.0.1 checked) This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
2002-06-11 10:20:31 +02:00
/*
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Save tables beeing updated in update_tables
update_table->shared is position for table
Don't use key read on tables that are updated
Big code cleanup/review before 4.0.2 release. (All commit emails since 4.0.1 checked) This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
2002-06-11 10:20:31 +02:00
*/
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
update.empty();
support of join view updateability (WL#1809)
2004-09-15 22:42:56 +02:00
for (table_ref= leaves; table_ref; table_ref= table_ref->next_leaf)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
fixed merged view fields names (BUG#5147) support of merged VIEW over several tables added (WL#1809)
2004-09-14 18:28:29 +02:00
/* TODO: add support of view of join support */
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE *table=table_ref->table;
if (tables_to_update & table->map)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE_LIST *tl= (TABLE_LIST*) thd->memdup((char*) table_ref,
sizeof(*tl));
if (!tl)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
DBUG_RETURN(1);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
update.link_in_list((byte*) tl, (byte**) &tl->next_local);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
tl->shared= table_count++;
table->no_keyread=1;
key_map with more that 64 bits width
2003-10-11 13:06:55 +02:00
table->used_keys.clear_all();
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
table->pos_in_table_list= tl;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
}
Two bug fixes
2003-04-11 19:09:24 +02:00
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
table_count= update.elements;
update_tables= (TABLE_LIST*) update.first;
First stage of table definition cache Split TABLE to TABLE and TABLE_SHARE (TABLE_SHARE is still allocated as part of table, will be fixed soon) Created Field::make_field() and made Field_num::make_field() to call this Added 'TABLE_SHARE->db' that points to database name; Changed all usage of table_cache_key as database name to use this instead Changed field->table_name to point to pointer to alias. This allows us to change alias for a table by just updating one pointer. Renamed TABLE_SHARE->real_name to table_name Renamed TABLE->table_name to alias Renamed TABLE_LIST->real_name to table_name
2005-01-06 12:00:13 +01:00
tmp_tables = (TABLE**) thd->calloc(sizeof(TABLE *) * table_count);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
tmp_table_param = (TMP_TABLE_PARAM*) thd->calloc(sizeof(TMP_TABLE_PARAM) *
table_count);
fields_for_table= (List_item **) thd->alloc(sizeof(List_item *) *
table_count);
values_for_table= (List_item **) thd->alloc(sizeof(List_item *) *
table_count);
fixed bug in determinating uncacheable queries new fatal_error interface to assign is_fatal_error and ne.report_error commant about Item_row
2003-01-30 21:15:44 +01:00
if (thd->is_fatal_error)
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(1);
for (i=0 ; i < table_count ; i++)
{
fields_for_table[i]= new List_item;
values_for_table[i]= new List_item;
}
fixed bug in determinating uncacheable queries new fatal_error interface to assign is_fatal_error and ne.report_error commant about Item_row
2003-01-30 21:15:44 +01:00
if (thd->is_fatal_error)
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(1);
/* Split fields into fields_for_table[] and values_by_table[] */
while ((item= (Item_field *) field_it++))
{
Item *value= value_it++;
uint offset= item->field->table->pos_in_table_list->shared;
fields_for_table[offset]->push_back(item);
values_for_table[offset]->push_back(value);
}
fixed bug in determinating uncacheable queries new fatal_error interface to assign is_fatal_error and ne.report_error commant about Item_row
2003-01-30 21:15:44 +01:00
if (thd->is_fatal_error)
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(1);
/* Allocate copy fields */
max_fields=0;
for (i=0 ; i < table_count ; i++)
set_if_bigger(max_fields, fields_for_table[i]->elements);
copy_field= new Copy_field[max_fields];
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
DBUG_RETURN(thd->is_fatal_error != 0);
}
Fix autoincrement for signed columns (Bug #1366) Fixed problem with char > 128 in QUOTE() function. (Bug #1868) Disable creation of symlinks if my_disable_symlink is set Fixed searching of TEXT with end space. (Bug #1651) Fixed caching bug in multi-table-update where same table was used twice. (Bug #1711) Fixed problem with UNIX_TIMESTAMP() for timestamps close to 0. (Bug #1998) Fixed timestamp.test
2003-12-12 21:26:58 +01:00
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
/*
Check if table is safe to update on fly
Fix autoincrement for signed columns (Bug #1366) Fixed problem with char > 128 in QUOTE() function. (Bug #1868) Disable creation of symlinks if my_disable_symlink is set Fixed searching of TEXT with end space. (Bug #1651) Fixed caching bug in multi-table-update where same table was used twice. (Bug #1711) Fixed problem with UNIX_TIMESTAMP() for timestamps close to 0. (Bug #1998) Fixed timestamp.test
2003-12-12 21:26:58 +01:00
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
SYNOPSIS
safe_update_on_fly()
thd Thread handler
join_tab How table is used in join
all_tables List of tables
Fix autoincrement for signed columns (Bug #1366) Fixed problem with char > 128 in QUOTE() function. (Bug #1868) Disable creation of symlinks if my_disable_symlink is set Fixed searching of TEXT with end space. (Bug #1651) Fixed caching bug in multi-table-update where same table was used twice. (Bug #1711) Fixed problem with UNIX_TIMESTAMP() for timestamps close to 0. (Bug #1998) Fixed timestamp.test
2003-12-12 21:26:58 +01:00
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
NOTES
We can update the first table in join on the fly if we know that
a row in this table will never be read twice. This is true under
the following conditions:
- We are doing a table scan and the data is in a separate file (MyISAM) or
if we don't update a clustered key.
- We are doing a range scan and we don't update the scan key or
the primary key for a clustered table handler.
- Table is not joined to itself.
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
This function gets information about fields to be updated from
the TABLE::write_set bitmap.
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-4.1-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines
2006-12-20 17:01:31 +01:00
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
WARNING
This code is a bit dependent of how make_join_readinfo() works.
RETURN
0 Not safe to update
1 Safe to update
*/
static bool safe_update_on_fly(THD *thd, JOIN_TAB *join_tab,
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
TABLE_LIST *table_ref, TABLE_LIST *all_tables)
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
{
TABLE *table= join_tab->table;
After merge fix.
2006-12-27 16:54:49 +01:00
if (unique_table(thd, table_ref, all_tables))
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
return 0;
switch (join_tab->type) {
case JT_SYSTEM:
case JT_CONST:
case JT_EQ_REF:
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-4.1-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines
2006-12-20 17:01:31 +01:00
return TRUE; // At most one matching row
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
case JT_REF:
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-4.1-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines
2006-12-20 17:01:31 +01:00
case JT_REF_OR_NULL:
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
return !is_key_used(table, join_tab->ref.key, table->write_set);
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
case JT_ALL:
/* If range search on index */
if (join_tab->quick)
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
return !join_tab->quick->is_keys_used(table->write_set);
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
/* If scanning in clustered key */
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
if ((table->file->ha_table_flags() & HA_PRIMARY_KEY_IN_READ_INDEX) &&
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-4.1-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines
2006-12-20 17:01:31 +01:00
table->s->primary_key < MAX_KEY)
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
return !is_key_used(table, table->s->primary_key, table->write_set);
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-4.1-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines
2006-12-20 17:01:31 +01:00
return TRUE;
BUG#21310 - Trees in SQL causing a "crashed" table with MyISAM storage engine An update that used a join of a table to itself and modified the table on one side of the join reported the table as crashed or updated wrong rows. Fixed by creating temporary table for self-joined multi update statement.
2006-12-20 16:05:35 +01:00
default:
break; // Avoid compler warning
Fix autoincrement for signed columns (Bug #1366) Fixed problem with char > 128 in QUOTE() function. (Bug #1868) Disable creation of symlinks if my_disable_symlink is set Fixed searching of TEXT with end space. (Bug #1651) Fixed caching bug in multi-table-update where same table was used twice. (Bug #1711) Fixed problem with UNIX_TIMESTAMP() for timestamps close to 0. (Bug #1998) Fixed timestamp.test
2003-12-12 21:26:58 +01:00
}
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-4.1-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines
2006-12-20 17:01:31 +01:00
return FALSE;
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/*
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
Initialize table for multi table
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
IMPLEMENTATION
- Update first table in join on the fly, if possible
- Create temporary tables to store changed values for all other tables
that are updated (and main_table if the above doesn't hold).
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
*/
bool
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
multi_update::initialize_tables(JOIN *join)
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE_LIST *table_ref;
DBUG_ENTER("initialize_tables");
if ((thd->options & OPTION_SAFE_UPDATES) && error_if_full_join(join))
DBUG_RETURN(1);
main_table=join->join_tab->table;
trans_safe= transactional_tables= main_table->file->has_transactions();
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
table_to_update= 0;
/* Create a temporary table for keys to all tables, except main table */
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
for (table_ref= update_tables; table_ref; table_ref= table_ref->next_local)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE *table=table_ref->table;
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
uint cnt= table_ref->shared;
after review changes
2004-03-20 12:36:26 +01:00
Item_field *ifield;
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
List<Item> temp_fields= *fields_for_table[cnt];
ORDER group;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
TMP_TABLE_PARAM *tmp_param;
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
table->mark_columns_needed_for_update();
BUG#21381 - Engine not notified about multi-table UPDATE IGNORE Though this is not storage engine specific problem, I was able to repeat this problem with BDB and NDB engines only. That was the reason to add a test case into ndb_update.test. As a result different bad things could happen. BDB has removed duplicate rows which is not expected. NDB returns an error. For multi table update notify storage engine about UPDATE IGNORE as it is done in single table UPDATE.
2006-10-05 15:23:53 +02:00
if (ignore)
table->file->extra(HA_EXTRA_IGNORE_DUP_KEY);
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
if (table == main_table) // First table in join
{
Merge mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.0-engines into mysql.com:/home/svoj/devel/mysql/BUG21310/mysql-5.1-engines
2006-12-29 14:33:34 +01:00
if (safe_update_on_fly(thd, join->join_tab, table_ref, all_tables))
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
{
table_to_update= main_table; // Update table on the fly
continue;
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
table->prepare_for_position();
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
tmp_param= tmp_table_param+cnt;
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
/*
Create a temporary table to store all fields that are changed for this
table. The first field in the temporary table is a pointer to the
original row so that we can find and update it
*/
/* ok to be on stack as this is not referenced outside of this func */
Field_string offset(table->file->ref_length, 0, "offset",
Table definition cache, part 2 The table opening process now works the following way: - Create common TABLE_SHARE object - Read the .frm file and unpack it into the TABLE_SHARE object - Create a TABLE object based on the information in the TABLE_SHARE object and open a handler to the table object Other noteworthy changes: - In TABLE_SHARE the most common strings are now LEX_STRING's - Better error message when table is not found - Variable table_cache is now renamed 'table_open_cache' - New variable 'table_definition_cache' that is the number of table defintions that will be cached - strxnmov() calls are now fixed to avoid overflows - strxnmov() will now always add one end \0 to result - engine objects are now created with a TABLE_SHARE object instead of a TABLE object. - After creating a field object one must call field->init(table) before using it - For a busy system this change will give you: - Less memory usage for table object - Faster opening of tables (if it's has been in use or is in table definition cache) - Allow you to cache many table definitions objects - Faster drop of table
2005-11-23 21:45:02 +01:00
&my_charset_bin);
offset.init(table);
Fixed bug #16281: Multi-table update broken in 5.0 on tables imported from 4.1 Mutli-table uses temporary table to store new values for fields. With the new values the rowid of the record to be updated is stored in a Field_string field. Table to be updated is set as source table of the rowid field. But when the temporary table creates the tmp field for the rowid field it converts it to a varstring field because the table to be updated was created by the v4.1. Due to this the stored rowids were broken and no records for update were found. The flag can_alter_field_type is added to Field_string class. When it is set to 0 the field won't be converted to varstring. The Field_string::type() function now always returns MYSQL_TYPE_STRING if can_alter_field_type is set to 0. The multi_update::initialize_tables() function now sets can_alter_field_type flag to 0 for the rowid fields denying conversion of the field to a varstring field.
2006-04-05 11:29:04 +02:00
/*
The field will be converted to varstring when creating tmp table if
table to be updated was created by mysql 4.1. Deny this.
*/
offset.can_alter_field_type= 0;
after review changes
2004-03-20 12:36:26 +01:00
if (!(ifield= new Item_field(((Field *) &offset))))
item_func.h: Fix for a bug with LEAST() in WHERE clause ha_innodb.cc: Fix for a configure bug multi_update.result, multi_update.test: Fix for the update with NULL's in the result set sql_update.cc: Fix for the update with NULL's in the result set Fix for the update with NULL's in the result set Fix for the update with NULL's in the result set
2003-08-20 14:33:21 +02:00
DBUG_RETURN(1);
after review changes
2004-03-20 12:36:26 +01:00
ifield->maybe_null= 0;
if (temp_fields.push_front(ifield))
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
DBUG_RETURN(1);
/* Make an unique key over the first field to avoid duplicated updates */
bzero((char*) &group, sizeof(group));
group.asc= 1;
group.item= (Item**) temp_fields.head_ref();
tmp_param->quick_group=1;
tmp_param->field_count=temp_fields.elements;
tmp_param->group_parts=1;
tmp_param->group_length= table->file->ref_length;
if (!(tmp_tables[cnt]=create_tmp_table(thd,
tmp_param,
temp_fields,
Merge with 4.0.12
2003-03-19 21:25:44 +01:00
(ORDER*) &group, 0, 0,
TMP_TABLE_ALL_COLUMNS,
fixed problem with reference on derived table fields (BUG#1031)
2003-08-12 14:04:49 +02:00
HA_POS_ERROR,
(char *) "")))
Allow optimzation of multi-table-update also for InnoDB tables MEMORY is alias for HEAP for CREATE TABLE ... TYPE=HEAP Fixed bug in multi-table-update where a row could be updated several times
2003-03-18 23:45:44 +01:00
DBUG_RETURN(1);
tmp_tables[cnt]->file->extra(HA_EXTRA_WRITE_CACHE);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(0);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
multi_update::~multi_update()
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE_LIST *table;
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
for (table= update_tables ; table; table= table->next_local)
BUG#21381 - Engine not notified about multi-table UPDATE IGNORE Though this is not storage engine specific problem, I was able to repeat this problem with BDB and NDB engines only. That was the reason to add a test case into ndb_update.test. As a result different bad things could happen. BDB has removed duplicate rows which is not expected. NDB returns an error. For multi table update notify storage engine about UPDATE IGNORE as it is done in single table UPDATE.
2006-10-05 15:23:53 +02:00
{
Fix autoincrement for signed columns (Bug #1366) Fixed problem with char > 128 in QUOTE() function. (Bug #1868) Disable creation of symlinks if my_disable_symlink is set Fixed searching of TEXT with end space. (Bug #1651) Fixed caching bug in multi-table-update where same table was used twice. (Bug #1711) Fixed problem with UNIX_TIMESTAMP() for timestamps close to 0. (Bug #1998) Fixed timestamp.test
2003-12-12 21:26:58 +01:00
table->table->no_keyread= table->table->no_cache= 0;
BUG#21381 - Engine not notified about multi-table UPDATE IGNORE Though this is not storage engine specific problem, I was able to repeat this problem with BDB and NDB engines only. That was the reason to add a test case into ndb_update.test. As a result different bad things could happen. BDB has removed duplicate rows which is not expected. NDB returns an error. For multi table update notify storage engine about UPDATE IGNORE as it is done in single table UPDATE.
2006-10-05 15:23:53 +02:00
if (ignore)
table->table->file->extra(HA_EXTRA_NO_IGNORE_DUP_KEY);
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
if (tmp_tables)
{
Copy arguments given to mysql_server_init() Made keybuff_size longlong (To make show variables work similar on 32 and 64 bit systems) Fixed some 'not initalized variable errors' in multi-table-update. Fixed memory leak in multi-table-update. Now all tests works under valgrind without any errors.
2002-12-05 15:38:49 +01:00
for (uint cnt = 0; cnt < table_count; cnt++)
{
if (tmp_tables[cnt])
{
free_tmp_table(thd, tmp_tables[cnt]);
tmp_table_param[cnt].cleanup();
}
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (copy_field)
delete [] copy_field;
Add warnings for single line inserts. To do this I had to convert count_cuted_fields to an enum (to be able to detect setting a NOT NULL field to NULL)
2003-10-11 22:26:39 +02:00
thd->count_cuted_fields= CHECK_FIELD_IGNORE; // Restore this setting
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (!trans_safe)
thd->options|=OPTION_STATUS_NO_TRANS_UPDATE;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
bool multi_update::send_data(List<Item> &not_used_values)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE_LIST *cur_table;
DBUG_ENTER("multi_update::send_data");
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
for (cur_table= update_tables; cur_table; cur_table= cur_table->next_local)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE *table= cur_table->table;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
uint offset= cur_table->shared;
Write binlog before commit when doing INSERT ... SELECT
2003-03-16 15:28:30 +01:00
/*
Check if we are using outer join and we didn't find the row
or if we have already updated this row in the previous call to this
function.
The same row may be presented here several times in a join of type
UPDATE t1 FROM t1,t2 SET t1.a=t2.a
In this case we will do the update for the first found row combination.
The join algorithm guarantees that we will not find the a row in
t1 several times.
*/
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (table->status & (STATUS_NULL_ROW | STATUS_UPDATED))
continue;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
/*
We can use compare_record() to optimize away updates if
the table handler is returning all columns OR if
if all updated columns are read
*/
Better fix for "Halloween" effect ...
2003-03-12 16:11:48 +01:00
if (table == table_to_update)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
bool can_compare_record;
can_compare_record= (!(table->file->ha_table_flags() &
HA_PARTIAL_COLUMN_READ) ||
bitmap_is_subset(table->write_set,
table->read_set));
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
table->status|= STATUS_UPDATED;
INSERT ... UPDATE ... VALUES()
2003-05-03 01:16:56 +02:00
store_record(table,record[1]);
Fix for bugs: #5860 "Multi-table UPDATE does not activate update triggers" #6812 "Triggers are not activated for INSERT ... SELECT" #8755 "Trigger is not activated by LOAD DATA". This patch also implements proper handling of triggers for special forms of insert like REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. Also now we don't call after trigger in case when we have failed to inserted/update or delete row. Trigger failure should stop statement execution. I have not properly tested handling of errors which happen inside of triggers in this patch, since it is simplier to do this once we will be able to access tables from triggers.
2005-05-24 20:19:33 +02:00
if (fill_record_n_invoke_before_triggers(thd, *fields_for_table[offset],
*values_for_table[offset], 0,
table->triggers,
TRG_EVENT_UPDATE))
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(1);
post-review fixes
2004-11-12 13:34:00 +01:00
Fix for found rows in multi-table updates
2002-12-18 18:00:00 +01:00
found++;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (!can_compare_record || compare_record(table))
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
int error;
After merge fixes Add support for warnings for prepare of prepared statements Fixed test to work with --ps-protocol Fixed some test results
2005-01-04 12:46:53 +01:00
if ((error= cur_table->view_check_option(thd, ignore)) !=
post review changes: CHECK OPTION moved to one function view name added to error messages
2004-09-29 15:35:01 +02:00
VIEW_CHECK_OK)
{
found--;
if (error == VIEW_CHECK_SKIP)
continue;
else if (error == VIEW_CHECK_ERROR)
DBUG_RETURN(1);
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (!updated++)
fixed two bugs in multi-table update
2002-08-06 20:24:12 +02:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/*
Inform the main table that we are going to update the table even
while we may be scanning it. This will flush the read cache
if it's used.
*/
main_table->file->extra(HA_EXTRA_PREPARE_FOR_UPDATE);
fixed two bugs in multi-table update
2002-08-06 20:24:12 +02:00
}
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
if ((error=table->file->ha_update_row(table->record[1],
table->record[0])))
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
updated--;
BUG#17138: Error in stored procedure due to fatal error when not really a fatal error. New handling of ignore error in place.
2006-06-20 02:56:50 +02:00
if (!ignore ||
BUG#17138: Crashes in stored procedure Last round of review fixes
2006-07-01 06:01:37 +02:00
table->file->is_fatal_error(error, HA_CHECK_DUP_KEY))
Bug#5553 - Multi table UPDATE IGNORE fails on dup key We don't want the update to abort when IGNORE is specified
2004-09-17 13:07:59 +02:00
{
Fixed BUG#13729 Stored procedures: packet error after exception handled Don't set thd->is_fatal_error in sql_update for duplicate key errors.
2005-11-23 11:56:53 +01:00
/*
BUG#17138: Error in stored procedure due to fatal error when not really a fatal error. New handling of ignore error in place.
2006-06-20 02:56:50 +02:00
If (ignore && error == is ignorable) we don't have to
Fixed BUG#13729 Stored procedures: packet error after exception handled Don't set thd->is_fatal_error in sql_update for duplicate key errors.
2005-11-23 11:56:53 +01:00
do anything; otherwise...
*/
BUG#17138: Crashes in stored procedure Last round of review fixes
2006-07-01 06:01:37 +02:00
if (table->file->is_fatal_error(error, HA_CHECK_DUP_KEY))
Fixed BUG#13729 Stored procedures: packet error after exception handled Don't set thd->is_fatal_error in sql_update for duplicate key errors.
2005-11-23 11:56:53 +01:00
thd->fatal_error(); /* Other handler errors are fatal */
Bug#5553 - Multi table UPDATE IGNORE fails on dup key We don't want the update to abort when IGNORE is specified
2004-09-17 13:07:59 +02:00
table->file->print_error(error,MYF(0));
DBUG_RETURN(1);
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
Fix for bugs: #5860 "Multi-table UPDATE does not activate update triggers" #6812 "Triggers are not activated for INSERT ... SELECT" #8755 "Trigger is not activated by LOAD DATA". This patch also implements proper handling of triggers for special forms of insert like REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. Also now we don't call after trigger in case when we have failed to inserted/update or delete row. Trigger failure should stop statement execution. I have not properly tested handling of errors which happen inside of triggers in this patch, since it is simplier to do this once we will be able to access tables from triggers.
2005-05-24 20:19:33 +02:00
else
{
if (!table->file->has_transactions())
thd->no_trans_update= 1;
if (table->triggers &&
table->triggers->process_triggers(thd, TRG_EVENT_UPDATE,
TRG_ACTION_AFTER, TRUE))
DBUG_RETURN(1);
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
}
else
{
int error;
TABLE *tmp_table= tmp_tables[offset];
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
table->file->position(table->record[0]);
post-review fixes
2004-11-12 13:34:00 +01:00
fill_record(thd, tmp_table->field+1, *values_for_table[offset], 1);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/* Store pointer to row */
memcpy((char*) tmp_table->field[0]->ptr,
(char*) table->file->ref, table->file->ref_length);
/* Write row, ignoring duplicated updates to a row */
Fix compiler warning
2006-01-04 09:16:34 +01:00
if ((error= tmp_table->file->ha_write_row(tmp_table->record[0])))
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
BUG#17138: Crashes in stored procedure Last round of review fixes
2006-07-01 06:01:37 +02:00
if (tmp_table->file->is_fatal_error(error, HA_CHECK_DUP) &&
Fix bug#15028 Multitable update returns different numbers of matched rows depending on table order multi_update::send_data() was counting updates, not updated rows. Thus if one record have several updates it will be counted several times in 'rows matched' but updated only once. multi_update::send_data() now counts only unique rows.
2005-12-01 21:22:20 +01:00
create_myisam_from_heap(thd, tmp_table,
tmp_table_param + offset, error, 1))
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
do_update=0;
DBUG_RETURN(1); // Not a table_is_full error
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
}
Fix bug#15028 Multitable update returns different numbers of matched rows depending on table order multi_update::send_data() was counting updates, not updated rows. Thus if one record have several updates it will be counted several times in 'rows matched' but updated only once. multi_update::send_data() now counts only unique rows.
2005-12-01 21:22:20 +01:00
else
found++;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(0);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
void multi_update::send_error(uint errcode,const char *err)
{
/* First send error what ever it is ... */
errors without code removed net_printf/send_error calls replaced by my_error family functions -1/1 (sent/unsent) error reporting removed (WL#2133)
2004-10-20 03:04:37 +02:00
my_error(errcode, MYF(0), err);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
/* If nothing updated return */
if (!updated)
return;
invalidation moved before tables unlocking
2002-04-28 23:33:52 +02:00
Big code cleanup/review before 4.0.2 release. (All commit emails since 4.0.1 checked) This had to be done now, before the 4.1 tree changes to much, to make it easy to propagate bug fixes to the 4.1 tree.
2002-06-11 10:20:31 +02:00
/* Something already updated so we have to invalidate cache */
invalidation moved before tables unlocking
2002-04-28 23:33:52 +02:00
query_cache_invalidate3(thd, update_tables, 1);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
/*
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
If all tables that has been updated are trans safe then just do rollback.
If not attempt to do remaining updates.
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
*/
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (trans_safe)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
ha_rollback_stmt(thd);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
else if (do_update && table_count > 1)
{
/* Add warning here */
VOID(do_updates(0));
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
int multi_update::do_updates(bool from_send_error)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
TABLE_LIST *cur_table;
int local_error;
ha_rows org_updated;
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
TABLE *table, *tmp_table;
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_ENTER("do_updates");
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
do_update= 0; // Don't retry this function
sql_delete.cc, sql_update.cc, sql_class.h: Simplification of the previous fix for multi-update/delete.
2003-12-12 22:14:59 +01:00
if (!found)
Many files: Fixed a bug causing a crash for multi-update/multi-delete with impossible where (bug #1860).
2003-12-11 23:55:48 +01:00
DBUG_RETURN(0);
VIEW two TABLE_LIST copy eliminated
2004-07-16 00:15:55 +02:00
for (cur_table= update_tables; cur_table; cur_table= cur_table->next_local)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
Ensure that install_test_db.sh script works with both 'make install' and 'make_binary_distribution' layouts Abort if we can't allocate memory for table cache Fix bug with multi-update-tables and BDB tables.
2004-03-12 00:10:22 +01:00
byte *ref_pos;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
bool can_compare_record;
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
table = cur_table->table;
Better fix for "Halloween" effect ...
2003-03-12 16:11:48 +01:00
if (table == table_to_update)
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
continue; // Already updated
org_updated= updated;
Ensure that install_test_db.sh script works with both 'make install' and 'make_binary_distribution' layouts Abort if we can't allocate memory for table cache Fix bug with multi-update-tables and BDB tables.
2004-03-12 00:10:22 +01:00
tmp_table= tmp_tables[cur_table->shared];
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
tmp_table->file->extra(HA_EXTRA_CACHE); // Change to read cache
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
(void) table->file->ha_rnd_init(0);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
table->file->extra(HA_EXTRA_NO_CACHE);
/*
Setup copy functions to copy fields from temporary table
*/
List_iterator_fast<Item> field_it(*fields_for_table[cur_table->shared]);
Field **field= tmp_table->field+1; // Skip row pointer
Copy_field *copy_field_ptr= copy_field, *copy_field_end;
for ( ; *field ; field++)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Item_field *item= (Item_field* ) field_it++;
(copy_field_ptr++)->set(item->field, *field, 0);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
copy_field_end=copy_field_ptr;
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
if ((local_error = tmp_table->file->ha_rnd_init(1)))
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
goto err;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
can_compare_record= (!(table->file->ha_table_flags() &
HA_PARTIAL_COLUMN_READ) ||
bitmap_is_subset(table->write_set,
table->read_set));
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
ref_pos= (byte*) tmp_table->field[0]->ptr;
for (;;)
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (thd->killed && trans_safe)
goto err;
if ((local_error=tmp_table->file->rnd_next(tmp_table->record[0])))
{
if (local_error == HA_ERR_END_OF_FILE)
break;
if (local_error == HA_ERR_RECORD_DELETED)
continue; // May happen on dup key
goto err;
}
if ((local_error= table->file->rnd_pos(table->record[0], ref_pos)))
goto err;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
table->status|= STATUS_UPDATED;
INSERT ... UPDATE ... VALUES()
2003-05-03 01:16:56 +02:00
store_record(table,record[1]);
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
/* Copy data from temporary table to current table */
for (copy_field_ptr=copy_field;
copy_field_ptr != copy_field_end;
copy_field_ptr++)
(*copy_field_ptr->do_copy)(copy_field_ptr);
Fix for bugs: #5860 "Multi-table UPDATE does not activate update triggers" #6812 "Triggers are not activated for INSERT ... SELECT" #8755 "Trigger is not activated by LOAD DATA". This patch also implements proper handling of triggers for special forms of insert like REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. Also now we don't call after trigger in case when we have failed to inserted/update or delete row. Trigger failure should stop statement execution. I have not properly tested handling of errors which happen inside of triggers in this patch, since it is simplier to do this once we will be able to access tables from triggers.
2005-05-24 20:19:33 +02:00
if (table->triggers &&
table->triggers->process_triggers(thd, TRG_EVENT_UPDATE,
TRG_ACTION_BEFORE, TRUE))
goto err2;
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
2006-06-04 17:52:22 +02:00
if (!can_compare_record || compare_record(table))
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
Fixed bug #25931. View check option clauses were ignored for updates of multi-table views when the updates could not be performed on fly and the rows to update had to be put into temporary tables first.
2007-02-07 23:41:57 +01:00
int error;
if ((error= cur_table->view_check_option(thd, ignore)) !=
VIEW_CHECK_OK)
{
if (error == VIEW_CHECK_SKIP)
continue;
else if (error == VIEW_CHECK_ERROR)
goto err;
}
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
if ((local_error=table->file->ha_update_row(table->record[1],
table->record[0])))
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
{
BUG#17138: Error in stored procedure due to fatal error when not really a fatal error. New handling of ignore error in place.
2006-06-20 02:56:50 +02:00
if (!ignore ||
BUG#17138: Crashes in stored procedure Last round of review fixes
2006-07-01 06:01:37 +02:00
table->file->is_fatal_error(local_error, HA_CHECK_DUP_KEY))
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
goto err;
}
updated++;
Fix for bugs: #5860 "Multi-table UPDATE does not activate update triggers" #6812 "Triggers are not activated for INSERT ... SELECT" #8755 "Trigger is not activated by LOAD DATA". This patch also implements proper handling of triggers for special forms of insert like REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. Also now we don't call after trigger in case when we have failed to inserted/update or delete row. Trigger failure should stop statement execution. I have not properly tested handling of errors which happen inside of triggers in this patch, since it is simplier to do this once we will be able to access tables from triggers.
2005-05-24 20:19:33 +02:00
if (table->triggers &&
table->triggers->process_triggers(thd, TRG_EVENT_UPDATE,
TRG_ACTION_AFTER, TRUE))
goto err2;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
}
if (updated != org_updated)
{
if (table->file->has_transactions())
query_id and my_xid -> ulonglong fix for binlog+autocommit+tclog comments, style fixes
2005-01-27 22:38:56 +01:00
transactional_tables= 1;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
else
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
trans_safe= 0; // Can't do safe rollback
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
(void) table->file->ha_rnd_end();
(void) tmp_table->file->ha_rnd_end();
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
DBUG_RETURN(0);
err:
if (!from_send_error)
After merge fixes Some bigger code changes was necessary becasue of the multi-table-update and the new HANDLER code
2004-10-07 09:50:13 +02:00
{
thd->fatal_error();
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
table->file->print_error(local_error,MYF(0));
After merge fixes Some bigger code changes was necessary becasue of the multi-table-update and the new HANDLER code
2004-10-07 09:50:13 +02:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
Fix for bugs: #5860 "Multi-table UPDATE does not activate update triggers" #6812 "Triggers are not activated for INSERT ... SELECT" #8755 "Trigger is not activated by LOAD DATA". This patch also implements proper handling of triggers for special forms of insert like REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. Also now we don't call after trigger in case when we have failed to inserted/update or delete row. Trigger failure should stop statement execution. I have not properly tested handling of errors which happen inside of triggers in this patch, since it is simplier to do this once we will be able to access tables from triggers.
2005-05-24 20:19:33 +02:00
err2:
handler interface cleanups: more logical table/index_flags return HA_ERR_WRONG_COMMAND instead of abstract methods where appropriate max_keys and other limits renamed to max_supported_keys/etc max_keys/etc are now wrappers to max_supported_keys/etc ha_index_init/ha_rnd_init/ha_index_end/ha_rnd_end are now wrappers to real {index,rnd}_{init,end} to enforce strict pairing
2004-06-23 12:29:05 +02:00
(void) table->file->ha_rnd_end();
(void) tmp_table->file->ha_rnd_end();
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (updated != org_updated)
{
if (table->file->has_transactions())
query_id and my_xid -> ulonglong fix for binlog+autocommit+tclog comments, style fixes
2005-01-27 22:38:56 +01:00
transactional_tables= 1;
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
else
trans_safe= 0;
}
DBUG_RETURN(1);
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New error messages Test of unsigned BIGINT values Fixes for queries-per-hour Cleanup of replication code (comments and portability fixes) Make most of the binary log code 4G clean Changed syntax for GRANT ... QUERIES PER HOUR
2002-01-29 17:32:16 +01:00
/* out: 1 if error, 0 if success */
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
bool multi_update::send_eof()
{
Precision Math implementation
2005-02-08 23:50:45 +01:00
char buff[STRING_BUFFER_USUAL_SIZE];
WL#3146 "less locking in auto_increment": this is a cleanup patch for our current auto_increment handling: new names for auto_increment variables in THD, new methods to manipulate them (see sql_class.h), some move into handler::, causing less backup/restore work when executing substatements. This makes the logic hopefully clearer, less work is is needed in mysql_insert(). By cleaning up, using different variables for different purposes (instead of one for 3 things...), we fix those bugs, which someone may want to fix in 5.0 too: BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate statement-based" BUG#20341 "stored function inserting into one auto_increment puts bad data in slave" BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE" (now if a row is updated, LAST_INSERT_ID() will return its id) and re-fixes: BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT" (already fixed differently by Ramil in 4.1) Test of documented behaviour of mysql_insert_id() (there was no test). The behaviour changes introduced are: - LAST_INSERT_ID() now returns "the first autogenerated auto_increment value successfully inserted", instead of "the first autogenerated auto_increment value if any row was successfully inserted", see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY UPDATE, see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() does not change if no autogenerated value was successfully inserted (it used to then be 0), see auto_increment.test. - if in INSERT SELECT no autogenerated value was successfully inserted, mysql_insert_id() now returns the id of the last inserted row (it already did this for INSERT VALUES), see mysql_client_test.c. - if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X (it already did this for INSERT VALUES), see mysql_client_test.c. - NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE, the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID influences not only the first row now. Additionally, when unlocking a table we check that the thread is not keeping a next_insert_id (as the table is unlocked that id is potentially out-of-date); forgetting about this next_insert_id is done in a new handler::ha_release_auto_increment(). Finally we prepare for engines capable of reserving finite-length intervals of auto_increment values: we store such intervals in THD. The next step (to be done by the replication team in 5.1) is to read those intervals from THD and actually store them in the statement-based binary log. NDB will be a good engine to test that.
2006-07-09 17:52:19 +02:00
ulonglong id;
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
thd->proc_info="updating reference tables";
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
/* Does updates for the last n - 1 tables, returns 0 if ok */
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
int local_error = (table_count) ? do_updates(0) : 0;
Portability fixes for Fortre C++ 5.0 (on Sun) in 32 and 64 bit modes.
2002-11-07 02:54:00 +01:00
thd->proc_info= "end";
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
ha_innodb.cc: Remove accidenatlly committed debug printfs when query cache is used sql_update.cc, sql_load.cc, sql_insert.cc, sql_delete.cc: For the transactional query cache algorithm to work we must invalidate the query cache in INSERT/DELETE/UPDATE before writing to the binlog or calling ha_autocommit_... Note that binlog writing may also call commit. The crucial thing is that the transaction which modified data must not be committed BEFORE the query cache is invalidated.
2003-05-26 18:10:43 +02:00
/* We must invalidate the query cache before binlog writing and
ha_autocommit_... */
if (updated)
{
query_cache_invalidate3(thd, update_tables, 1);
}
New error messages Test of unsigned BIGINT values Fixes for queries-per-hour Cleanup of replication code (comments and portability fixes) Make most of the binary log code 4G clean Changed syntax for GRANT ... QUERIES PER HOUR
2002-01-29 17:32:16 +01:00
/*
Write the SQL statement to the binlog if we updated
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
rows and we succeeded or if we updated some non
Fix fur BUG#13348: "multi-table updates and deletes are not logged if no rows were affected". Not fixed in 4.1 as not critical. Also I'm correcting error checking of multi-UPDATE/DELETE when it comes to binlogging, to make it consistent with when we rollback the statement.
2005-09-20 17:41:47 +02:00
transactional tables.
New error messages Test of unsigned BIGINT values Fixes for queries-per-hour Cleanup of replication code (comments and portability fixes) Make most of the binary log code 4G clean Changed syntax for GRANT ... QUERIES PER HOUR
2002-01-29 17:32:16 +01:00
*/
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
Fix fur BUG#13348: "multi-table updates and deletes are not logged if no rows were affected". Not fixed in 4.1 as not critical. Also I'm correcting error checking of multi-UPDATE/DELETE when it comes to binlogging, to make it consistent with when we rollback the statement.
2005-09-20 17:41:47 +02:00
if ((local_error == 0) || (updated && !trans_safe))
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
{
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (mysql_bin_log.is_open())
{
Fix fur BUG#13348: "multi-table updates and deletes are not logged if no rows were affected". Not fixed in 4.1 as not critical. Also I'm correcting error checking of multi-UPDATE/DELETE when it comes to binlogging, to make it consistent with when we rollback the statement.
2005-09-20 17:41:47 +02:00
if (local_error == 0)
Fix for BUG#2083 "EE_ error codes (EE_DELETE, EE_WRITE) end up in the binlog, making slave stop". The problem was that during execution of the command on the master, an error can occur (for example, not space left on device, then mysqld waits and when there is space it completes successfully: so finally it worked but the error EE_WRITE remains in thd->net.last_errno and thd->net.last_error). To know if finally the command succeeded, we test the 'error' variable in every place, and if it shows no failure we reset thd->net.last_err* using the function THD::clear_error() which is backported from 4.1. A new test to see if now only real errors get to the binlog (note: the test uses "rm"). Also a bit of memory free/alloc saving in log_event.cc (do not free the whole mem_root after every query in the slave SQL thread: we can keep the initial block of it; which will be freed when the thread terminates).
2003-12-16 11:10:50 +01:00
thd->clear_error();
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
if (thd->binlog_query(THD::ROW_QUERY_TYPE,
thd->query, thd->query_length,
transactional_tables, FALSE) &&
trans_safe)
{
Fixed some bugs from last multi-table-update push. More tests for multi-table-update & timestamp handling
2002-11-30 23:48:35 +01:00
local_error= 1; // Rollback update
WL#1012: All changes as one single changeset. This includes both code and test cases.
2005-12-22 06:39:02 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
}
query_id and my_xid -> ulonglong fix for binlog+autocommit+tclog comments, style fixes
2005-01-27 22:38:56 +01:00
if (!transactional_tables)
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
thd->options|=OPTION_STATUS_NO_TRANS_UPDATE;
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (transactional_tables)
{
Fixed some bugs from last multi-table-update push. More tests for multi-table-update & timestamp handling
2002-11-30 23:48:35 +01:00
if (ha_autocommit_or_rollback(thd, local_error != 0))
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
local_error=1;
}
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
if (local_error > 0) // if the above log write did not fail ...
{
/* Safety: If we haven't got an error before (should not happen) */
my_message(ER_UNKNOWN_ERROR, "An error occured in multi-table update",
MYF(0));
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
return TRUE;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
New multi-table-update code New (simpler) internal timestamp handling. More debuging to heap tables. Small cleanups to multi-table-delete false -> 0 and true -> 1 (We should use TRUE and FALSE)
2002-11-29 15:40:18 +01:00
WL#3146 "less locking in auto_increment": this is a cleanup patch for our current auto_increment handling: new names for auto_increment variables in THD, new methods to manipulate them (see sql_class.h), some move into handler::, causing less backup/restore work when executing substatements. This makes the logic hopefully clearer, less work is is needed in mysql_insert(). By cleaning up, using different variables for different purposes (instead of one for 3 things...), we fix those bugs, which someone may want to fix in 5.0 too: BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate statement-based" BUG#20341 "stored function inserting into one auto_increment puts bad data in slave" BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE" (now if a row is updated, LAST_INSERT_ID() will return its id) and re-fixes: BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT" (already fixed differently by Ramil in 4.1) Test of documented behaviour of mysql_insert_id() (there was no test). The behaviour changes introduced are: - LAST_INSERT_ID() now returns "the first autogenerated auto_increment value successfully inserted", instead of "the first autogenerated auto_increment value if any row was successfully inserted", see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY UPDATE, see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() does not change if no autogenerated value was successfully inserted (it used to then be 0), see auto_increment.test. - if in INSERT SELECT no autogenerated value was successfully inserted, mysql_insert_id() now returns the id of the last inserted row (it already did this for INSERT VALUES), see mysql_client_test.c. - if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X (it already did this for INSERT VALUES), see mysql_client_test.c. - NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE, the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID influences not only the first row now. Additionally, when unlocking a table we check that the thread is not keeping a next_insert_id (as the table is unlocked that id is potentially out-of-date); forgetting about this next_insert_id is done in a new handler::ha_release_auto_increment(). Finally we prepare for engines capable of reserving finite-length intervals of auto_increment values: we store such intervals in THD. The next step (to be done by the replication team in 5.1) is to read those intervals from THD and actually store them in the statement-based binary log. NDB will be a good engine to test that.
2006-07-09 17:52:19 +02:00
id= thd->arg_of_last_insert_id_function ?
thd->first_successful_insert_id_in_prev_stmt : 0;
Fix for the bug #841: wrong number of warnings
2003-07-14 12:32:31 +02:00
sprintf(buff, ER(ER_UPDATE_INFO), (ulong) found, (ulong) updated,
(ulong) thd->cuted_fields);
WL#1567: Add ROW_COUNT() Function to retrieve count of rows updated/inserted/deleted
2004-05-04 13:45:20 +02:00
thd->row_count_func=
(thd->client_capabilities & CLIENT_FOUND_ROWS) ? found : updated;
WL#3146 "less locking in auto_increment": this is a cleanup patch for our current auto_increment handling: new names for auto_increment variables in THD, new methods to manipulate them (see sql_class.h), some move into handler::, causing less backup/restore work when executing substatements. This makes the logic hopefully clearer, less work is is needed in mysql_insert(). By cleaning up, using different variables for different purposes (instead of one for 3 things...), we fix those bugs, which someone may want to fix in 5.0 too: BUG#20339 "stored procedure using LAST_INSERT_ID() does not replicate statement-based" BUG#20341 "stored function inserting into one auto_increment puts bad data in slave" BUG#19243 "wrong LAST_INSERT_ID() after ON DUPLICATE KEY UPDATE" (now if a row is updated, LAST_INSERT_ID() will return its id) and re-fixes: BUG#6880 "LAST_INSERT_ID() value changes during multi-row INSERT" (already fixed differently by Ramil in 4.1) Test of documented behaviour of mysql_insert_id() (there was no test). The behaviour changes introduced are: - LAST_INSERT_ID() now returns "the first autogenerated auto_increment value successfully inserted", instead of "the first autogenerated auto_increment value if any row was successfully inserted", see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() returns the id of the updated row if ON DUPLICATE KEY UPDATE, see auto_increment.test. Same for mysql_insert_id(), see mysql_client_test.c. - LAST_INSERT_ID() does not change if no autogenerated value was successfully inserted (it used to then be 0), see auto_increment.test. - if in INSERT SELECT no autogenerated value was successfully inserted, mysql_insert_id() now returns the id of the last inserted row (it already did this for INSERT VALUES), see mysql_client_test.c. - if INSERT SELECT uses LAST_INSERT_ID(X), mysql_insert_id() now returns X (it already did this for INSERT VALUES), see mysql_client_test.c. - NDB now behaves like other engines wrt SET INSERT_ID: with INSERT IGNORE, the id passed in SET INSERT_ID is re-used until a row succeeds; SET INSERT_ID influences not only the first row now. Additionally, when unlocking a table we check that the thread is not keeping a next_insert_id (as the table is unlocked that id is potentially out-of-date); forgetting about this next_insert_id is done in a new handler::ha_release_auto_increment(). Finally we prepare for engines capable of reserving finite-length intervals of auto_increment values: we store such intervals in THD. The next step (to be done by the replication team in 5.1) is to read those intervals from THD and actually store them in the statement-based binary log. NDB will be a good engine to test that.
2006-07-09 17:52:19 +02:00
::send_ok(thd, (ulong) thd->row_count_func, id, buff);
now my_printf_error is not better then my_error, but my_error call is shorter used only one implementation of format parser of (printf) fixed multistatement
2004-11-13 18:35:51 +01:00
return FALSE;
This is a large push. Included are : * multi-table updates * new paid feature for limiting number of queries per hour for users * optional syntax for multi-table deletes * optimization for SQL_CALC_FOUND_ROWS * a small addition for CREATE .. SELECT that will be of future use I know that all this will require many additions to documentation, which I have not done, but I am at Arjen's disposal to help him document all this.
2001-12-26 15:49:10 +01:00
}
Reference in a new issue Copy permalink