mariadb/storage/federated/ha_federated.h
unknown 5099033c26 WL#3138: Maria - fast "SELECT COUNT(*) FROM t;" and "CHECKSUM TABLE t"
Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation
Fixed wrong call to strmake
Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert
Allow storing year 2155 in year field
When running with purify/valgrind avoid copying structures over themself
Added hook 'trnnam_end_trans_hook' that is called when transaction ends
Added trn->used_tables that is used to an entry for all tables used by transaction
Fixed that ndb doesn't crash on duplicate key error when start_bulk_insert/end_bulk_insert are not called


include/maria.h:
  Added argument to maria_end_bulk_insert() to know if the table will be deleted after the operation
include/my_tree.h:
  Added macro 'reset_free_element()' to be able to ignore calls to the external free function.
  Is used to optimize end-bulk-insert in case of failures, in which case we don't want write the remaining keys in the tree
mysql-test/install_test_db.sh:
  Upgrade to new mysql_install_db options
mysql-test/r/maria-mvcc.result:
  New tests
mysql-test/r/maria.result:
  New tests
mysql-test/suite/ndb/r/ndb_auto_increment.result:
  Fixed error message now when bulk insert is not always called
mysql-test/suite/ndb/t/ndb_auto_increment.test:
  Fixed error message now when bulk insert is not always called
mysql-test/t/maria-mvcc.test:
  Added testing of versioning of count(*)
mysql-test/t/maria-page-checksum.test:
  Added comment
mysql-test/t/maria.test:
  More tests
mysys/hash.c:
  Code style change
sql/field.cc:
  Allow storing year 2155 in year field
sql/ha_ndbcluster.cc:
  Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/ha_ndbcluster.h:
  Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/ha_partition.cc:
  Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/ha_partition.h:
  Added new argument to end_bulk_insert() to signal if the bulk insert should ignored
sql/handler.cc:
  Don't call get_dup_key() if there is no table object. This can happen if the handler generates a duplicate key error on commit
sql/handler.h:
  Added new argument to end_bulk_insert() to signal if the bulk insert should ignored (ie, the table will be deleted)
sql/item.cc:
  Style fix
  Removed compiler warning
sql/log_event.cc:
  Added new argument to ha_end_bulk_insert()
sql/log_event_old.cc:
  Added new argument to ha_end_bulk_insert()
sql/mysqld.cc:
  Removed compiler warning
sql/protocol.cc:
  Added DBUG
sql/sql_class.cc:
  Added DBUG
  Fixed wrong call to strmake
sql/sql_insert.cc:
  Don't call bulk insert in case of inserting only one row (speed optimization as starting/stopping bulk insert involves a lot of if's)
  Added new argument to ha_end_bulk_insert()
sql/sql_load.cc:
  Added new argument to ha_end_bulk_insert()
sql/sql_parse.cc:
  Style fixes
  Avoid goto in common senario
sql/sql_select.cc:
  When running with purify/valgrind avoid copying structures over themself.  This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings
sql/sql_select.h:
  Avoid copying structures over themself.  This is not a real bug in itself, but it's a waste of cycles and causes valgrind warnings
sql/sql_table.cc:
  Call HA_EXTRA_PREPARE_FOR_DROP if table created by ALTER TABLE is going to be dropped
  Added new argument to ha_end_bulk_insert()
storage/archive/ha_archive.cc:
  Added new argument to end_bulk_insert()
storage/archive/ha_archive.h:
  Added new argument to end_bulk_insert()
storage/federated/ha_federated.cc:
  Added new argument to end_bulk_insert()
storage/federated/ha_federated.h:
  Added new argument to end_bulk_insert()
storage/maria/Makefile.am:
  Added ma_state.c and ma_state.h
storage/maria/ha_maria.cc:
  Versioning of count(*) and checksum
  - share->state.state is now assumed to be correct, not handler->state
  - Call _ma_setup_live_state() in external lock to get count(*)/checksum versioning. In case of
    not versioned and not concurrent insertable table, file->s->state.state contains the correct state information
  
  Other things:
  - file->s -> share
  - Added DBUG_ASSERT() for unlikely case
  - Optimized end_bulk_insert() to not write anything if table is going to be deleted (as in failed alter table)
  - Indentation changes in external_lock becasue of removed 'goto' caused a big conflict even if very little was changed
storage/maria/ha_maria.h:
  New argument to end_bulk_insert()
storage/maria/ma_blockrec.c:
  Update for versioning of count(*) and checksum
  Keep share->state.state.data_file_length up to date (not info->state->data_file_length)
  Moved _ma_block_xxxx_status() and maria_versioning() functions to ma_state.c
storage/maria/ma_check.c:
  Update and use share->state.state instead of info->state
  info->s to share
  Update info->state at end of repair
  Call _ma_reset_state() to update share->state_history at end of repair
storage/maria/ma_checkpoint.c:
  Call _ma_remove_not_visible_states() on checkpoint to clean up not visible state history from tables
storage/maria/ma_close.c:
  Remember state history for running transaction even if table is closed
storage/maria/ma_commit.c:
  Ensure we always call trnman_commit_trn() even if other calls fails. If we don't do that, the translog and state structures will not be freed
storage/maria/ma_delete.c:
  Versioning of count(*) and checksum:
  - Always update info->state->checksum and info->state->records
storage/maria/ma_delete_all.c:
  Versioning of count(*) and checksum:
  - Ensure that share->state.state is updated, as here is where we store the primary information
storage/maria/ma_dynrec.c:
  Use lock_key_trees instead of concurrent_insert to check if trees should be locked.
  This allows us to lock trees both for concurrent_insert and for index versioning.
storage/maria/ma_extra.c:
  Versioning of count(*) and checksum:
  - Use share->state.state instead of info->state
  - share->concurrent_insert -> share->non_transactional_concurrent_insert
  - Don't update share->state.state from info->state if transactional table
  
  Optimization:
  - Don't flush io_cache or bitmap if we are using FLUSH_IGNORE_CHANGED
storage/maria/ma_info.c:
  Get most state information from current state
storage/maria/ma_init.c:
  Add hash table and free function to store states for closed tables
  Install hook for transaction commit/rollback to update history state
storage/maria/ma_key_recover.c:
  Versioning of count(*) and checksum:
  - Use share->state.state instead of info->state
storage/maria/ma_locking.c:
  Versioning of count(*) and checksum:
  - Call virtual functions (if exists) to restore/update status
  - Move _ma_xxx_status() functions to ma_state.c
  
  info->s -> share
storage/maria/ma_open.c:
  Versioning of count(*) and checksum:
  - For not transactional tables, set info->state to point to new allocated state structure.
  - Initialize new info->state_start variable that points to state at start of transaction
  - Copy old history states from hash table (maria_stored_states) first time the table is opened
  - Split flag share->concurrent_insert to non_transactional_concurrent_insert & lock_key_tree
  - For now, only enable versioning of tables without keys (to be fixed in soon!)
  - Added new virtual function to restore status in maria_lock_database)
  
  More DBUG
storage/maria/ma_page.c:
  Versioning of count(*) and checksum:
  - Use share->state.state instead of info->state
  - Modify share->state.state.key_file_length under share->intern_lock
storage/maria/ma_range.c:
  Versioning of count(*) and checksum:
  - Lock trees based on share->lock_key_trees
  
  info->s -> share
storage/maria/ma_recovery.c:
  Versioning of count(*) and checksum:
  - Use share->state.state instead of info->state
  - Update state information on close and when reenabling logging
storage/maria/ma_rkey.c:
  Versioning of count(*) and checksum:
  - Lock trees based on share->lock_key_trees
storage/maria/ma_rnext.c:
  Versioning of count(*) and checksum:
  - Lock trees based on share->lock_key_trees
storage/maria/ma_rnext_same.c:
  Versioning of count(*) and checksum:
  - Lock trees based on share->lock_key_trees
  - Only skip rows based on file length if non_transactional_concurrent_insert is set
storage/maria/ma_rprev.c:
  Versioning of count(*) and checksum:
  - Lock trees based on share->lock_key_trees
storage/maria/ma_rsame.c:
  Versioning of count(*) and checksum:
  - Lock trees based on share->lock_key_trees
storage/maria/ma_sort.c:
  Use share->state.state instead of info->state
  Fixed indentation
storage/maria/ma_static.c:
  Added maria_stored_state
storage/maria/ma_update.c:
  Versioning of count(*) and checksum:
  - Always update info->state->checksum and info->state->records
  - Remove optimization for index file update as it doesn't work for transactional tables
storage/maria/ma_write.c:
  Versioning of count(*) and checksum:
  - Always update info->state->checksum and info->state->records
storage/maria/maria_def.h:
  Move MARIA_STATUS_INFO to ma_state.h
  
  Changes to MARIA_SHARE:
  - Added state_history to store count(*)/checksum states
  - Added in_trans as counter if table is used by running transactions
  - Split concurrent_insert into lock_key_trees and on_transactional_concurrent_insert.
  - Added virtual function lock_restore_status
  
  Changes to MARIA_HA:
  - save_state -> state_save
  - Added state_start to store state at start of transaction
storage/maria/maria_pack.c:
  Versioning of count(*) and checksum:
  - Use share->state.state instead of info->state
  
  Indentation fixes
storage/maria/trnman.c:
  Added hook 'trnnam_end_trans_hook' that is called when transaction ends
  Added trn->used_tables that is used to an entry for all tables used by transaction
  More DBUG
  Changed return type of trnman_end_trn() to my_bool
  Added trnman_get_min_trid() to get minimum trid in use.
  Added trnman_exists_active_transactions() to check if there exist a running transaction started between two commit id
storage/maria/trnman.h:
  Added 'used_tables'
  Moved all pointers into same groups to get better memory alignment
storage/maria/trnman_public.h:
  Added prototypes for new functions and variables
  Chagned return type of trnman_end_trn() to my_bool
storage/myisam/ha_myisam.cc:
  Added argument to end_bulk_insert() if operation should be aborted
storage/myisam/ha_myisam.h:
  Added argument to end_bulk_insert() if operation should be aborted
storage/maria/ma_state.c:
  Functions to handle state of count(*) and checksum
storage/maria/ma_state.h:
  Structures and declarations to handle state of count(*) and checksum
2008-05-29 18:33:33 +03:00

261 lines
9.5 KiB
C++

/* Copyright (C) 2003 MySQL AB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
/*
Please read ha_exmple.cc before reading this file.
Please keep in mind that the federated storage engine implements all methods
that are required to be implemented. handler.h has a full list of methods
that you can implement.
*/
#ifdef USE_PRAGMA_INTERFACE
#pragma interface /* gcc class implementation */
#endif
#include <mysql.h>
/*
handler::print_error has a case statement for error numbers.
This value is (10000) is far out of range and will envoke the
default: case.
(Current error range is 120-159 from include/my_base.h)
*/
#define HA_FEDERATED_ERROR_WITH_REMOTE_SYSTEM 10000
#define FEDERATED_QUERY_BUFFER_SIZE STRING_BUFFER_USUAL_SIZE * 5
#define FEDERATED_RECORDS_IN_RANGE 2
#define FEDERATED_MAX_KEY_LENGTH 3500 // Same as innodb
/*
FEDERATED_SHARE is a structure that will be shared amoung all open handlers
The example implements the minimum of what you will probably need.
*/
typedef struct st_federated_share {
MEM_ROOT mem_root;
bool parsed;
/* this key is unique db/tablename */
const char *share_key;
/*
the primary select query to be used in rnd_init
*/
char *select_query;
/*
remote host info, parse_url supplies
*/
char *server_name;
char *connection_string;
char *scheme;
char *connect_string;
char *hostname;
char *username;
char *password;
char *database;
char *table_name;
char *table;
char *socket;
char *sport;
int share_key_length;
ushort port;
uint table_name_length, server_name_length, connect_string_length, use_count;
pthread_mutex_t mutex;
THR_LOCK lock;
} FEDERATED_SHARE;
/*
Class definition for the storage engine
*/
class ha_federated: public handler
{
THR_LOCK_DATA lock; /* MySQL lock */
FEDERATED_SHARE *share; /* Shared lock info */
MYSQL *mysql; /* MySQL connection */
MYSQL_RES *stored_result;
uint fetch_num; // stores the fetch num
MYSQL_ROW_OFFSET current_position; // Current position used by ::position()
int remote_error_number;
char remote_error_buf[FEDERATED_QUERY_BUFFER_SIZE];
bool ignore_duplicates, replace_duplicates;
bool insert_dup_update;
DYNAMIC_STRING bulk_insert;
private:
/*
return 0 on success
return errorcode otherwise
*/
uint convert_row_to_internal_format(uchar *buf, MYSQL_ROW row,
MYSQL_RES *result);
bool create_where_from_key(String *to, KEY *key_info,
const key_range *start_key,
const key_range *end_key,
bool records_in_range, bool eq_range);
int stash_remote_error();
bool append_stmt_insert(String *query);
int read_next(uchar *buf, MYSQL_RES *result);
int index_read_idx_with_result_set(uchar *buf, uint index,
const uchar *key,
uint key_len,
ha_rkey_function find_flag,
MYSQL_RES **result);
int real_query(const char *query, uint length);
int real_connect();
public:
ha_federated(handlerton *hton, TABLE_SHARE *table_arg);
~ha_federated() {}
/* The name that will be used for display purposes */
const char *table_type() const { return "FEDERATED"; }
/*
Next pointer used in transaction
*/
ha_federated *trx_next;
/*
The name of the index type that will be used for display
don't implement this method unless you really have indexes
*/
// perhaps get index type
const char *index_type(uint inx) { return "REMOTE"; }
const char **bas_ext() const;
/*
This is a list of flags that says what the storage engine
implements. The current table flags are documented in
handler.h
*/
ulonglong table_flags() const
{
/* fix server to be able to get remote server table flags */
return (HA_PRIMARY_KEY_IN_READ_INDEX | HA_FILE_BASED
| HA_REC_NOT_IN_SEQ | HA_AUTO_PART_KEY | HA_CAN_INDEX_BLOBS |
HA_BINLOG_ROW_CAPABLE | HA_BINLOG_STMT_CAPABLE |
HA_NO_PREFIX_CHAR_KEYS | HA_PRIMARY_KEY_REQUIRED_FOR_DELETE |
HA_NO_TRANSACTIONS /* until fixed by WL#2952 */ |
HA_PARTIAL_COLUMN_READ | HA_NULL_IN_KEY);
}
/*
This is a bitmap of flags that says how the storage engine
implements indexes. The current index flags are documented in
handler.h. If you do not implement indexes, just return zero
here.
part is the key part to check. First key part is 0
If all_parts it's set, MySQL want to know the flags for the combined
index up to and including 'part'.
*/
/* fix server to be able to get remote server index flags */
ulong index_flags(uint inx, uint part, bool all_parts) const
{
return (HA_READ_NEXT | HA_READ_RANGE | HA_READ_AFTER_KEY);
}
uint max_supported_record_length() const { return HA_MAX_REC_LENGTH; }
uint max_supported_keys() const { return MAX_KEY; }
uint max_supported_key_parts() const { return MAX_REF_PARTS; }
uint max_supported_key_length() const { return FEDERATED_MAX_KEY_LENGTH; }
uint max_supported_key_part_length() const { return FEDERATED_MAX_KEY_LENGTH; }
/*
Called in test_quick_select to determine if indexes should be used.
Normally, we need to know number of blocks . For federated we need to
know number of blocks on remote side, and number of packets and blocks
on the network side (?)
Talk to Kostja about this - how to get the
number of rows * ...
disk scan time on other side (block size, size of the row) + network time ...
The reason for "records * 1000" is that such a large number forces
this to use indexes "
*/
double scan_time()
{
DBUG_PRINT("info", ("records %lu", (ulong) stats.records));
return (double)(stats.records*1000);
}
/*
The next method will never be called if you do not implement indexes.
*/
double read_time(uint index, uint ranges, ha_rows rows)
{
/*
Per Brian, this number is bugus, but this method must be implemented,
and at a later date, he intends to document this issue for handler code
*/
return (double) rows / 20.0+1;
}
const key_map *keys_to_use_for_scanning() { return &key_map_full; }
/*
Everything below are methods that we implment in ha_federated.cc.
Most of these methods are not obligatory, skip them and
MySQL will treat them as not implemented
*/
int open(const char *name, int mode, uint test_if_locked); // required
int close(void); // required
void start_bulk_insert(ha_rows rows);
int end_bulk_insert(bool abort);
int write_row(uchar *buf);
int update_row(const uchar *old_data, uchar *new_data);
int delete_row(const uchar *buf);
int index_init(uint keynr, bool sorted);
ha_rows estimate_rows_upper_bound();
int index_read(uchar *buf, const uchar *key,
uint key_len, enum ha_rkey_function find_flag);
int index_read_idx(uchar *buf, uint idx, const uchar *key,
uint key_len, enum ha_rkey_function find_flag);
int index_next(uchar *buf);
int index_end();
int read_range_first(const key_range *start_key,
const key_range *end_key,
bool eq_range, bool sorted);
int read_range_next();
/*
unlike index_init(), rnd_init() can be called two times
without rnd_end() in between (it only makes sense if scan=1).
then the second call should prepare for the new table scan
(e.g if rnd_init allocates the cursor, second call should
position it to the start of the table, no need to deallocate
and allocate it again
*/
int rnd_init(bool scan); //required
int rnd_end();
int rnd_next(uchar *buf); //required
int rnd_pos(uchar *buf, uchar *pos); //required
void position(const uchar *record); //required
int info(uint); //required
int extra(ha_extra_function operation);
void update_auto_increment(void);
int repair(THD* thd, HA_CHECK_OPT* check_opt);
int optimize(THD* thd, HA_CHECK_OPT* check_opt);
int delete_all_rows(void);
int create(const char *name, TABLE *form,
HA_CREATE_INFO *create_info); //required
ha_rows records_in_range(uint inx, key_range *start_key,
key_range *end_key);
uint8 table_cache_type() { return HA_CACHE_TBL_NOCACHE; }
THR_LOCK_DATA **store_lock(THD *thd, THR_LOCK_DATA **to,
enum thr_lock_type lock_type); //required
bool get_error_message(int error, String *buf);
int external_lock(THD *thd, int lock_type);
int connection_commit();
int connection_rollback();
int connection_autocommit(bool state);
int execute_simple_query(const char *query, int len);
int reset(void);
};