mirror/mariadb
1
0
Fork 0
mirror of https://github.com/MariaDB/server.git synced 2025-01-17 20:42:30 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
mariadb/storage/myisam/mi_packrec.c
unknown f252f9248a WL#3817: Simplify string / memory area types and make things more consistent (first part) 
The following type conversions was done:

- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t

Removed declaration of byte, gptr, my_string, my_size_t and size_s. 

Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
  instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
  as this requires fewer casts in the code and is more in line with how the
  standard functions work.
- Added extra length argument to dirname_part() to return the length of the
  created string.
- Changed (at least) following functions to take uchar* as argument:
  - db_dump()
  - my_net_write()
  - net_write_command()
  - net_store_data()
  - DBUG_DUMP()
  - decimal2bin() & bin2decimal()
- Changed my_compress() and my_uncompress() to use size_t. Changed one
  argument to my_uncompress() from a pointer to a value as we only return
  one value (makes function easier to use).
- Changed type of 'pack_data' argument to packfrm() to avoid casts.
- Changed in readfrm() and writefrom(), ha_discover and handler::discover()
  the type for argument 'frmdata' to uchar** to avoid casts.
- Changed most Field functions to use uchar* instead of char* (reduced a lot of
  casts).
- Changed field->val_xxx(xxx, new_ptr) to take const pointers.

Other changes:
- Removed a lot of not needed casts
- Added a few new cast required by other changes
- Added some cast to my_multi_malloc() arguments for safety (as string lengths
  needs to be uint, not size_t).
- Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done
  explicitely as this conflict was often hided by casting the function to
  hash_get_key).
- Changed some buffers to memory regions to uchar* to avoid casts.
- Changed some string lengths from uint to size_t.
- Changed field->ptr to be uchar* instead of char*. This allowed us to
  get rid of a lot of casts.
- Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar
- Include zlib.h in some files as we needed declaration of crc32()
- Changed MY_FILE_ERROR to be (size_t) -1.
- Changed many variables to hold the result of my_read() / my_write() to be
  size_t. This was needed to properly detect errors (which are
  returned as (size_t) -1).
- Removed some very old VMS code
- Changed packfrm()/unpackfrm() to not be depending on uint size
  (portability fix)
- Removed windows specific code to restore cursor position as this
  causes slowdown on windows and we should not mix read() and pread()
  calls anyway as this is not thread safe. Updated function comment to
  reflect this. Changed function that depended on original behavior of
  my_pwrite() to itself restore the cursor position (one such case).
- Added some missing checking of return value of malloc().
- Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow.
- Changed type of table_def::m_size from my_size_t to ulong to reflect that
  m_size is the number of elements in the array, not a string/memory
  length.
- Moved THD::max_row_length() to table.cc (as it's not depending on THD).
  Inlined max_row_length_blob() into this function.
- More function comments
- Fixed some compiler warnings when compiled without partitions.
- Removed setting of LEX_STRING() arguments in declaration (portability fix).
- Some trivial indentation/variable name changes.
- Some trivial code simplifications:
  - Replaced some calls to alloc_root + memcpy to use
    strmake_root()/strdup_root().
  - Changed some calls from memdup() to strmake() (Safety fix)
  - Simpler loops in client-simple.c


BitKeeper/etc/ignore:
  added libmysqld/ha_ndbcluster_cond.cc
  ---
  added debian/defs.mk debian/control
client/completion_hash.cc:
  Remove not needed casts
client/my_readline.h:
  Remove some old types
client/mysql.cc:
  Simplify types
client/mysql_upgrade.c:
  Remove some old types
  Update call to dirname_part
client/mysqladmin.cc:
  Remove some old types
client/mysqlbinlog.cc:
  Remove some old types
  Change some buffers to be uchar to avoid casts
client/mysqlcheck.c:
  Remove some old types
client/mysqldump.c:
  Remove some old types
  Remove some not needed casts
  Change some string lengths to size_t
client/mysqlimport.c:
  Remove some old types
client/mysqlshow.c:
  Remove some old types
client/mysqlslap.c:
  Remove some old types
  Remove some not needed casts
client/mysqltest.c:
  Removed some old types
  Removed some not needed casts
  Updated hash-get-key function arguments
  Updated parameters to dirname_part()
client/readline.cc:
  Removed some old types
  Removed some not needed casts
  Changed some string lengths to use size_t
client/sql_string.cc:
  Removed some old types
dbug/dbug.c:
  Removed some old types
  Changed some string lengths to use size_t
  Changed some prototypes to avoid casts
extra/comp_err.c:
  Removed some old types
extra/innochecksum.c:
  Removed some old types
extra/my_print_defaults.c:
  Removed some old types
extra/mysql_waitpid.c:
  Removed some old types
extra/perror.c:
  Removed some old types
extra/replace.c:
  Removed some old types
  Updated parameters to dirname_part()
extra/resolve_stack_dump.c:
  Removed some old types
extra/resolveip.c:
  Removed some old types
include/config-win.h:
  Removed some old types
include/decimal.h:
  Changed binary strings to be uchar* instead of char*
include/ft_global.h:
  Removed some old types
include/hash.h:
  Removed some old types
include/heap.h:
  Removed some old types
  Changed records_under_level to be 'ulong' instead of 'uint' to clarify usage of variable
include/keycache.h:
  Removed some old types
include/m_ctype.h:
  Removed some old types
  Changed some string lengths to use size_t
  Changed character length functions to return uint
  unsigned char -> uchar
include/m_string.h:
  Removed some old types
  Changed some string lengths to use size_t
include/my_alloc.h:
  Changed some string lengths to use size_t
include/my_base.h:
  Removed some old types
include/my_dbug.h:
  Removed some old types
  Changed some string lengths to use size_t
  Changed db_dump() to take uchar * as argument for memory to reduce number of casts in usage
include/my_getopt.h:
  Removed some old types
include/my_global.h:
  Removed old types:
  my_size_t -> size_t
  byte -> uchar
  gptr -> uchar *
include/my_list.h:
  Removed some old types
include/my_nosys.h:
  Removed some old types
include/my_pthread.h:
  Removed some old types
include/my_sys.h:
  Removed some old types
  Changed MY_FILE_ERROR to be in line with new definitions of my_write()/my_read()
  Changed some string lengths to use size_t
  my_malloc() / my_free() now uses void *
  Updated parameters to dirname_part() & my_uncompress()
include/my_tree.h:
  Removed some old types
include/my_trie.h:
  Removed some old types
include/my_user.h:
  Changed some string lengths to use size_t
include/my_vle.h:
  Removed some old types
include/my_xml.h:
  Removed some old types
  Changed some string lengths to use size_t
include/myisam.h:
  Removed some old types
include/myisammrg.h:
  Removed some old types
include/mysql.h:
  Removed some old types
  Changed byte streams to use uchar* instead of char*
include/mysql_com.h:
  Removed some old types
  Changed some string lengths to use size_t
  Changed some buffers to be uchar* to avoid casts
include/queues.h:
  Removed some old types
include/sql_common.h:
  Removed some old types
include/sslopt-longopts.h:
  Removed some old types
include/violite.h:
  Removed some old types
  Changed some string lengths to use size_t
libmysql/client_settings.h:
  Removed some old types
libmysql/libmysql.c:
  Removed some old types
libmysql/manager.c:
  Removed some old types
libmysqld/emb_qcache.cc:
  Removed some old types
libmysqld/emb_qcache.h:
  Removed some old types
libmysqld/lib_sql.cc:
  Removed some old types
  Removed some not needed casts
  Changed some buffers to be uchar* to avoid casts
  true -> TRUE, false -> FALSE
mysys/array.c:
  Removed some old types
mysys/charset.c:
  Changed some string lengths to use size_t
mysys/checksum.c:
  Include zlib to get definition for crc32
  Removed some old types
mysys/default.c:
  Removed some old types
  Changed some string lengths to use size_t
mysys/default_modify.c:
  Changed some string lengths to use size_t
  Removed some not needed casts
mysys/hash.c:
  Removed some old types
  Changed some string lengths to use size_t
  Note: Prototype of hash_key() has changed which may cause problems if client uses hash_init() with a cast for the hash-get-key function.
  hash_element now takes 'ulong' as the index type (cleanup)
mysys/list.c:
  Removed some old types
mysys/mf_cache.c:
  Changed some string lengths to use size_t
mysys/mf_dirname.c:
  Removed some old types
  Changed some string lengths to use size_t
  Added argument to dirname_part() to avoid calculation of length for 'to'
mysys/mf_fn_ext.c:
  Removed some old types
  Updated parameters to dirname_part()
mysys/mf_format.c:
  Removed some old types
  Changed some string lengths to use size_t
mysys/mf_getdate.c:
  Removed some old types
mysys/mf_iocache.c:
  Removed some old types
  Changed some string lengths to use size_t
  Changed calculation of 'max_length' to be done the same way in all functions
mysys/mf_iocache2.c:
  Removed some old types
  Changed some string lengths to use size_t
  Clean up comments
  Removed not needed indentation
mysys/mf_keycache.c:
  Removed some old types
mysys/mf_keycaches.c:
  Removed some old types
mysys/mf_loadpath.c:
  Removed some old types
mysys/mf_pack.c:
  Removed some old types
  Changed some string lengths to use size_t
  Removed some not needed casts
  Removed very old VMS code
  Updated parameters to dirname_part()
  Use result of dirnam_part() to remove call to strcat()
mysys/mf_path.c:
  Removed some old types
mysys/mf_radix.c:
  Removed some old types
mysys/mf_same.c:
  Removed some old types
mysys/mf_sort.c:
  Removed some old types
mysys/mf_soundex.c:
  Removed some old types
mysys/mf_strip.c:
  Removed some old types
mysys/mf_tempdir.c:
  Removed some old types
mysys/mf_unixpath.c:
  Removed some old types
mysys/mf_wfile.c:
  Removed some old types
mysys/mulalloc.c:
  Removed some old types
mysys/my_alloc.c:
  Removed some old types
  Changed some string lengths to use size_t
  Use void* as type for allocated memory area
  Removed some not needed casts
  Changed argument 'Size' to 'length' according coding guidelines
mysys/my_chsize.c:
  Changed some buffers to be uchar* to avoid casts
mysys/my_compress.c:
  More comments
  Removed some old types
  Changed string lengths to use size_t
  Changed arguments to my_uncompress() to make them easier to understand
  Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix)
  Changed type of 'pack_data' argument to packfrm() to avoid casts.
mysys/my_conio.c:
  Changed some string lengths to use size_t
mysys/my_create.c:
  Removed some old types
mysys/my_div.c:
  Removed some old types
mysys/my_error.c:
  Removed some old types
mysys/my_fopen.c:
  Removed some old types
mysys/my_fstream.c:
  Removed some old types
  Changed some string lengths to use size_t
  writen -> written
mysys/my_getopt.c:
  Removed some old types
mysys/my_getwd.c:
  Removed some old types
  More comments
mysys/my_init.c:
  Removed some old types
mysys/my_largepage.c:
  Removed some old types
  Changed some string lengths to use size_t
mysys/my_lib.c:
  Removed some old types
mysys/my_lockmem.c:
  Removed some old types
mysys/my_malloc.c:
  Removed some old types
  Changed malloc(), free() and related functions to use void *
  Changed all functions to use size_t
mysys/my_memmem.c:
  Indentation cleanup
mysys/my_once.c:
  Removed some old types
  Changed malloc(), free() and related functions to use void *
mysys/my_open.c:
  Removed some old types
mysys/my_pread.c:
  Removed some old types
  Changed all functions to use size_t
  Added comment for how my_pread() / my_pwrite() are supposed to work.
  Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe.
  (If we ever would really need this, it should be enabled only with a flag argument)
mysys/my_quick.c:
  Removed some old types
  Changed all functions to use size_t
mysys/my_read.c:
  Removed some old types
  Changed all functions to use size_t
mysys/my_realloc.c:
  Removed some old types
  Use void* as type for allocated memory area
  Changed all functions to use size_t
mysys/my_static.c:
  Removed some old types
mysys/my_static.h:
  Removed some old types
mysys/my_vle.c:
  Removed some old types
mysys/my_wincond.c:
  Removed some old types
mysys/my_windac.c:
  Removed some old types
mysys/my_write.c:
  Removed some old types
  Changed all functions to use size_t
mysys/ptr_cmp.c:
  Removed some old types
  Changed all functions to use size_t
mysys/queues.c:
  Removed some old types
mysys/safemalloc.c:
  Removed some old types
  Changed malloc(), free() and related functions to use void *
  Changed all functions to use size_t
mysys/string.c:
  Removed some old types
  Changed all functions to use size_t
mysys/testhash.c:
  Removed some old types
mysys/thr_alarm.c:
  Removed some old types
mysys/thr_lock.c:
  Removed some old types
mysys/tree.c:
  Removed some old types
mysys/trie.c:
  Removed some old types
mysys/typelib.c:
  Removed some old types
plugin/daemon_example/daemon_example.cc:
  Removed some old types
regex/reginit.c:
  Removed some old types
server-tools/instance-manager/buffer.cc:
  Changed some string lengths to use size_t
  Changed buffer to be of type uchar*
server-tools/instance-manager/buffer.h:
  Changed some string lengths to use size_t
  Changed buffer to be of type uchar*
server-tools/instance-manager/commands.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Changed buffer to be of type uchar*
server-tools/instance-manager/instance_map.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Changed buffer to be of type uchar*
server-tools/instance-manager/instance_options.cc:
  Changed buffer to be of type uchar*
  Replaced alloc_root + strcpy() with strdup_root()
server-tools/instance-manager/mysql_connection.cc:
  Changed buffer to be of type uchar*
server-tools/instance-manager/options.cc:
  Removed some old types
server-tools/instance-manager/parse.cc:
  Changed some string lengths to use size_t
server-tools/instance-manager/parse.h:
  Removed some old types
  Changed some string lengths to use size_t
server-tools/instance-manager/protocol.cc:
  Changed some buffers to be uchar* to avoid casts
  Changed some string lengths to use size_t
server-tools/instance-manager/protocol.h:
  Changed some string lengths to use size_t
server-tools/instance-manager/user_map.cc:
  Removed some old types
  Changed some string lengths to use size_t
sql/derror.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Changed some string lengths to use size_t
sql/discover.cc:
  Changed in readfrm() and writefrom() the type for argument 'frmdata' to uchar** to avoid casts
  Changed some string lengths to use size_t
  Changed some buffers to be uchar* to avoid casts
sql/event_data_objects.cc:
  Removed some old types
  Added missing casts for alloc() and sprintf()
sql/event_db_repository.cc:
  Changed some buffers to be uchar* to avoid casts
  Added missing casts for sprintf()
sql/event_queue.cc:
  Removed some old types
sql/field.cc:
  Removed some old types
  Changed memory buffers to be uchar*
  Changed some string lengths to use size_t
  Removed a lot of casts
  Safety fix in Field_blob::val_decimal() to not access zero pointer
sql/field.h:
  Removed some old types
  Changed memory buffers to be uchar* (except of store() as this would have caused too many other changes). 
  Changed some string lengths to use size_t
  Removed some not needed casts
  Changed val_xxx(xxx, new_ptr) to take const pointers
sql/field_conv.cc:
  Removed some old types
  Added casts required because memory area pointers are now uchar*
sql/filesort.cc:
  Initalize variable that was used unitialized in error conditions
sql/gen_lex_hash.cc:
  Removed some old types
  Changed memory buffers to be uchar*
  Changed some string lengths to use size_t
  Removed a lot of casts
  Safety fix in Field_blob::val_decimal() to not access zero pointer
sql/gstream.h:
  Added required cast
sql/ha_ndbcluster.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Changed some buffers to be uchar* to avoid casts
  Added required casts
  Removed some not needed casts
sql/ha_ndbcluster.h:
  Removed some old types
sql/ha_ndbcluster_binlog.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Replaced sql_alloc() + memcpy() + set end 0 with sql_strmake()
  Changed some string lengths to use size_t
  Added missing casts for alloc() and sprintf()
sql/ha_ndbcluster_binlog.h:
  Removed some old types
sql/ha_ndbcluster_cond.cc:
  Removed some old types
  Removed some not needed casts
sql/ha_ndbcluster_cond.h:
  Removed some old types
sql/ha_partition.cc:
  Removed some old types
  Changed prototype for change_partition() to avoid casts
sql/ha_partition.h:
  Removed some old types
sql/handler.cc:
  Removed some old types
  Changed some string lengths to use size_t
sql/handler.h:
  Removed some old types
  Changed some string lengths to use size_t
  Changed type for 'frmblob' parameter for discover() and ha_discover() to get fewer casts
sql/hash_filo.h:
  Removed some old types
  Changed all functions to use size_t
sql/hostname.cc:
  Removed some old types
sql/item.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Use strmake() instead of memdup() to create a null terminated string.
  Updated calls to new Field()
sql/item.h:
  Removed some old types
  Changed malloc(), free() and related functions to use void *
  Changed some buffers to be uchar* to avoid casts
sql/item_cmpfunc.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
sql/item_cmpfunc.h:
  Removed some old types
sql/item_create.cc:
  Removed some old types
sql/item_func.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
  Added test for failing alloc() in init_result_field()
  Remove old confusing comment
  Fixed compiler warning
sql/item_func.h:
  Removed some old types
sql/item_row.cc:
  Removed some old types
sql/item_row.h:
  Removed some old types
sql/item_strfunc.cc:
  Include zlib (needed becasue we call crc32)
  Removed some old types
sql/item_strfunc.h:
  Removed some old types
  Changed some types to match new function prototypes
sql/item_subselect.cc:
  Removed some old types
sql/item_subselect.h:
  Removed some old types
sql/item_sum.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/item_sum.h:
  Removed some old types
sql/item_timefunc.cc:
  Removed some old types
  Changed some string lengths to use size_t
sql/item_timefunc.h:
  Removed some old types
sql/item_xmlfunc.cc:
  Changed some string lengths to use size_t
sql/item_xmlfunc.h:
  Removed some old types
sql/key.cc:
  Removed some old types
  Removed some not needed casts
sql/lock.cc:
  Removed some old types
  Added some cast to my_multi_malloc() arguments for safety
sql/log.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Changed some buffers to be uchar* to avoid casts
  Changed usage of pwrite() to not assume it holds the cursor position for the file
  Made usage of my_read() safer
sql/log_event.cc:
  Removed some old types
  Added checking of return value of malloc() in pack_info()
  Changed some buffers to be uchar* to avoid casts
  Removed some 'const' to avoid casts
  Added missing casts for alloc() and sprintf()
  Added required casts
  Removed some not needed casts
  Added some cast to my_multi_malloc() arguments for safety
sql/log_event.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
sql/log_event_old.cc:
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/log_event_old.h:
  Changed some buffers to be uchar* to avoid casts
sql/mf_iocache.cc:
  Removed some old types
sql/my_decimal.cc:
  Changed memory area to use uchar*
sql/my_decimal.h:
  Changed memory area to use uchar*
sql/mysql_priv.h:
  Removed some old types
  Changed malloc(), free() and related functions to use void *
  Changed some string lengths to use size_t
  Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid long overflow
  Changed some buffers to be uchar* to avoid casts
sql/mysqld.cc:
  Removed some old types
sql/net_serv.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Changed some buffers to be uchar* to avoid casts
  Ensure that vio_read()/vio_write() return values are stored in a size_t variable
  Removed some not needed casts
sql/opt_range.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/opt_range.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
sql/opt_sum.cc:
  Removed some old types
  Removed some not needed casts
sql/parse_file.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Changed alloc_root + memcpy + set end 0 -> strmake_root()
sql/parse_file.h:
  Removed some old types
sql/partition_info.cc:
  Removed some old types
  Added missing casts for alloc()
  Changed some buffers to be uchar* to avoid casts
sql/partition_info.h:
  Changed some buffers to be uchar* to avoid casts
sql/protocol.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/protocol.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Changed some string lengths to use size_t
sql/records.cc:
  Removed some old types
sql/repl_failsafe.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Added required casts
sql/rpl_filter.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Changed some string lengths to use size_t
sql/rpl_filter.h:
  Changed some string lengths to use size_t
sql/rpl_injector.h:
  Removed some old types
sql/rpl_record.cc:
  Removed some old types
  Removed some not needed casts
  Changed some buffers to be uchar* to avoid casts
sql/rpl_record.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
sql/rpl_record_old.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/rpl_record_old.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid cast
sql/rpl_rli.cc:
  Removed some old types
sql/rpl_tblmap.cc:
  Removed some old types
sql/rpl_tblmap.h:
  Removed some old types
sql/rpl_utility.cc:
  Removed some old types
sql/rpl_utility.h:
  Removed some old types
  Changed type of m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length
sql/set_var.cc:
  Removed some old types
  Updated parameters to dirname_part()
sql/set_var.h:
  Removed some old types
sql/slave.cc:
  Removed some old types
  Changed some string lengths to use size_t
sql/slave.h:
  Removed some old types
sql/sp.cc:
  Removed some old types
  Added missing casts for printf()
sql/sp.h:
  Removed some old types
  Updated hash-get-key function arguments
sql/sp_cache.cc:
  Removed some old types
  Added missing casts for printf()
  Updated hash-get-key function arguments
sql/sp_head.cc:
  Removed some old types
  Added missing casts for alloc() and printf()
  Added required casts
  Updated hash-get-key function arguments
sql/sp_head.h:
  Removed some old types
sql/sp_pcontext.cc:
  Removed some old types
sql/sp_pcontext.h:
  Removed some old types
sql/sql_acl.cc:
  Removed some old types
  Changed some string lengths to use size_t
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
  Added required casts
sql/sql_analyse.cc:
  Changed some buffers to be uchar* to avoid casts
sql/sql_analyse.h:
  Changed some buffers to be uchar* to avoid casts
sql/sql_array.h:
  Removed some old types
sql/sql_base.cc:
  Removed some old types
  Updated hash-get-key function arguments
sql/sql_binlog.cc:
  Removed some old types
  Added missing casts for printf()
sql/sql_cache.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Removed some not needed casts
  Changed some string lengths to use size_t
sql/sql_cache.h:
  Removed some old types
  Removed reference to not existing function cache_key()
  Updated hash-get-key function arguments
sql/sql_class.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Added missing casts for alloc()
  Updated hash-get-key function arguments
  Moved THD::max_row_length() to table.cc (as it's not depending on THD)
  Removed some not needed casts
sql/sql_class.h:
  Removed some old types
  Changed malloc(), free() and related functions to use void *
  Removed some not needed casts
  Changed some string lengths to use size_t
  Moved max_row_length and max_row_length_blob() to table.cc, as they are not depending on THD
sql/sql_connect.cc:
  Removed some old types
  Added required casts
sql/sql_db.cc:
  Removed some old types
  Removed some not needed casts
  Added some cast to my_multi_malloc() arguments for safety
  Added missing casts for alloc()
sql/sql_delete.cc:
  Removed some old types
sql/sql_handler.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Added some cast to my_multi_malloc() arguments for safety
sql/sql_help.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/sql_insert.cc:
  Removed some old types
  Added missing casts for alloc() and printf()
sql/sql_lex.cc:
  Removed some old types
sql/sql_lex.h:
  Removed some old types
  Removed some not needed casts
sql/sql_list.h:
  Removed some old types
  Removed some not needed casts
sql/sql_load.cc:
  Removed some old types
  Removed compiler warning
sql/sql_manager.cc:
  Removed some old types
sql/sql_map.cc:
  Removed some old types
sql/sql_map.h:
  Removed some old types
sql/sql_olap.cc:
  Removed some old types
sql/sql_parse.cc:
  Removed some old types
  Trivial move of code lines to make things more readable
  Changed some string lengths to use size_t
  Added missing casts for alloc()
sql/sql_partition.cc:
  Removed some old types
  Removed compiler warnings about not used functions
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/sql_partition.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
sql/sql_plugin.cc:
  Removed some old types
  Added missing casts for alloc()
  Updated hash-get-key function arguments
sql/sql_prepare.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Added missing casts for alloc() and printf()
sql-common/client.c:
  Removed some old types
  Changed some memory areas to use uchar*
sql-common/my_user.c:
  Changed some string lengths to use size_t
sql-common/pack.c:
  Changed some buffers to be uchar* to avoid casts
sql/sql_repl.cc:
  Added required casts
  Changed some buffers to be uchar* to avoid casts
  Changed some string lengths to use size_t
sql/sql_select.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some old types
sql/sql_select.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
sql/sql_servers.cc:
  Removed some old types
  Updated hash-get-key function arguments
sql/sql_show.cc:
  Removed some old types
  Added missing casts for alloc()
  Removed some not needed casts
sql/sql_string.cc:
  Removed some old types
  Added required casts
sql/sql_table.cc:
  Removed some old types
  Removed compiler warning about not used variable
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
sql/sql_test.cc:
  Removed some old types
sql/sql_trigger.cc:
  Removed some old types
  Added missing casts for alloc()
sql/sql_udf.cc:
  Removed some old types
  Updated hash-get-key function arguments
sql/sql_union.cc:
  Removed some old types
sql/sql_update.cc:
  Removed some old types
  Removed some not needed casts
sql/sql_view.cc:
  Removed some old types
sql/sql_yacc.yy:
  Removed some old types
  Changed some string lengths to use size_t
  Added missing casts for alloc()
sql/stacktrace.c:
  Removed some old types
sql/stacktrace.h:
  Removed some old types
sql/structs.h:
  Removed some old types
sql/table.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Changed some buffers to be uchar* to avoid casts
  Removed setting of LEX_STRING() arguments in declaration
  Added required casts
  More function comments
  Moved max_row_length() here from sql_class.cc/sql_class.h
sql/table.h:
  Removed some old types
  Changed some string lengths to use size_t
sql/thr_malloc.cc:
  Use void* as type for allocated memory area
  Changed all functions to use size_t
sql/tzfile.h:
  Changed some buffers to be uchar* to avoid casts
sql/tztime.cc:
  Changed some buffers to be uchar* to avoid casts
  Updated hash-get-key function arguments
  Added missing casts for alloc()
  Removed some not needed casts
sql/uniques.cc:
  Removed some old types
  Removed some not needed casts
sql/unireg.cc:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
  Removed some not needed casts
  Added missing casts for alloc()
storage/archive/archive_reader.c:
  Removed some old types
storage/archive/azio.c:
  Removed some old types
  Removed some not needed casts
storage/archive/ha_archive.cc:
  Removed some old types
  Changed type for 'frmblob' in archive_discover() to match handler
  Updated hash-get-key function arguments
  Removed some not needed casts
storage/archive/ha_archive.h:
  Removed some old types
storage/blackhole/ha_blackhole.cc:
  Removed some old types
storage/blackhole/ha_blackhole.h:
  Removed some old types
storage/csv/ha_tina.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Changed some buffers to be uchar* to avoid casts
storage/csv/ha_tina.h:
  Removed some old types
  Removed some not needed casts
storage/csv/transparent_file.cc:
  Removed some old types
  Changed type of 'bytes_read' to be able to detect read errors
  Fixed indentation
storage/csv/transparent_file.h:
  Removed some old types
storage/example/ha_example.cc:
  Removed some old types
  Updated hash-get-key function arguments
storage/example/ha_example.h:
  Removed some old types
storage/federated/ha_federated.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Removed some not needed casts
storage/federated/ha_federated.h:
  Removed some old types
storage/heap/_check.c:
  Changed some buffers to be uchar* to avoid casts
storage/heap/_rectest.c:
  Removed some old types
storage/heap/ha_heap.cc:
  Removed some old types
storage/heap/ha_heap.h:
  Removed some old types
storage/heap/heapdef.h:
  Removed some old types
storage/heap/hp_block.c:
  Removed some old types
  Changed some string lengths to use size_t
storage/heap/hp_clear.c:
  Removed some old types
storage/heap/hp_close.c:
  Removed some old types
storage/heap/hp_create.c:
  Removed some old types
storage/heap/hp_delete.c:
  Removed some old types
storage/heap/hp_hash.c:
  Removed some old types
storage/heap/hp_info.c:
  Removed some old types
storage/heap/hp_open.c:
  Removed some old types
storage/heap/hp_rfirst.c:
  Removed some old types
storage/heap/hp_rkey.c:
  Removed some old types
storage/heap/hp_rlast.c:
  Removed some old types
storage/heap/hp_rnext.c:
  Removed some old types
storage/heap/hp_rprev.c:
  Removed some old types
storage/heap/hp_rrnd.c:
  Removed some old types
storage/heap/hp_rsame.c:
  Removed some old types
storage/heap/hp_scan.c:
  Removed some old types
storage/heap/hp_test1.c:
  Removed some old types
storage/heap/hp_test2.c:
  Removed some old types
storage/heap/hp_update.c:
  Removed some old types
storage/heap/hp_write.c:
  Removed some old types
  Changed some string lengths to use size_t
storage/innobase/handler/ha_innodb.cc:
  Removed some old types
  Updated hash-get-key function arguments
  Added missing casts for alloc() and printf()
  Removed some not needed casts
storage/innobase/handler/ha_innodb.h:
  Removed some old types
storage/myisam/ft_boolean_search.c:
  Removed some old types
storage/myisam/ft_nlq_search.c:
  Removed some old types
storage/myisam/ft_parser.c:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
storage/myisam/ft_static.c:
  Removed some old types
storage/myisam/ft_stopwords.c:
  Removed some old types
storage/myisam/ft_update.c:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
storage/myisam/ftdefs.h:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
storage/myisam/fulltext.h:
  Removed some old types
storage/myisam/ha_myisam.cc:
  Removed some old types
storage/myisam/ha_myisam.h:
  Removed some old types
storage/myisam/mi_cache.c:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
storage/myisam/mi_check.c:
  Removed some old types
storage/myisam/mi_checksum.c:
  Removed some old types
storage/myisam/mi_close.c:
  Removed some old types
storage/myisam/mi_create.c:
  Removed some old types
storage/myisam/mi_delete.c:
  Removed some old types
storage/myisam/mi_delete_all.c:
  Removed some old types
storage/myisam/mi_dynrec.c:
  Removed some old types
storage/myisam/mi_extra.c:
  Removed some old types
storage/myisam/mi_key.c:
  Removed some old types
storage/myisam/mi_locking.c:
  Removed some old types
storage/myisam/mi_log.c:
  Removed some old types
storage/myisam/mi_open.c:
  Removed some old types
  Removed some not needed casts
  Check argument of my_write()/my_pwrite() in functions returning int
  Added casting of string lengths to size_t
storage/myisam/mi_packrec.c:
  Removed some old types
  Changed some buffers to be uchar* to avoid casts
storage/myisam/mi_page.c:
  Removed some old types
storage/myisam/mi_preload.c:
  Removed some old types
storage/myisam/mi_range.c:
  Removed some old types
storage/myisam/mi_rfirst.c:
  Removed some old types
storage/myisam/mi_rkey.c:
  Removed some old types
storage/myisam/mi_rlast.c:
  Removed some old types
storage/myisam/mi_rnext.c:
  Removed some old types
storage/myisam/mi_rnext_same.c:
  Removed some old types
storage/myisam/mi_rprev.c:
  Removed some old types
storage/myisam/mi_rrnd.c:
  Removed some old types
storage/myisam/mi_rsame.c:
  Removed some old types
storage/myisam/mi_rsamepos.c:
  Removed some old types
storage/myisam/mi_scan.c:
  Removed some old types
storage/myisam/mi_search.c:
  Removed some old types
storage/myisam/mi_static.c:
  Removed some old types
storage/myisam/mi_statrec.c:
  Removed some old types
storage/myisam/mi_test1.c:
  Removed some old types
storage/myisam/mi_test2.c:
  Removed some old types
storage/myisam/mi_test3.c:
  Removed some old types
storage/myisam/mi_unique.c:
  Removed some old types
storage/myisam/mi_update.c:
  Removed some old types
storage/myisam/mi_write.c:
  Removed some old types
storage/myisam/myisam_ftdump.c:
  Removed some old types
storage/myisam/myisamchk.c:
  Removed some old types
storage/myisam/myisamdef.h:
  Removed some old types
storage/myisam/myisamlog.c:
  Removed some old types
  Indentation fix
storage/myisam/myisampack.c:
  Removed some old types
storage/myisam/rt_index.c:
  Removed some old types
storage/myisam/rt_split.c:
  Removed some old types
storage/myisam/sort.c:
  Removed some old types
storage/myisam/sp_defs.h:
  Removed some old types
storage/myisam/sp_key.c:
  Removed some old types
storage/myisammrg/ha_myisammrg.cc:
  Removed some old types
storage/myisammrg/ha_myisammrg.h:
  Removed some old types
storage/myisammrg/myrg_close.c:
  Removed some old types
storage/myisammrg/myrg_def.h:
  Removed some old types
storage/myisammrg/myrg_delete.c:
  Removed some old types
storage/myisammrg/myrg_open.c:
  Removed some old types
  Updated parameters to dirname_part()
storage/myisammrg/myrg_queue.c:
  Removed some old types
storage/myisammrg/myrg_rfirst.c:
  Removed some old types
storage/myisammrg/myrg_rkey.c:
  Removed some old types
storage/myisammrg/myrg_rlast.c:
  Removed some old types
storage/myisammrg/myrg_rnext.c:
  Removed some old types
storage/myisammrg/myrg_rnext_same.c:
  Removed some old types
storage/myisammrg/myrg_rprev.c:
  Removed some old types
storage/myisammrg/myrg_rrnd.c:
  Removed some old types
storage/myisammrg/myrg_rsame.c:
  Removed some old types
storage/myisammrg/myrg_update.c:
  Removed some old types
storage/myisammrg/myrg_write.c:
  Removed some old types
storage/ndb/include/util/ndb_opts.h:
  Removed some old types
storage/ndb/src/cw/cpcd/main.cpp:
  Removed some old types
storage/ndb/src/kernel/vm/Configuration.cpp:
  Removed some old types
storage/ndb/src/mgmclient/main.cpp:
  Removed some old types
storage/ndb/src/mgmsrv/InitConfigFileParser.cpp:
  Removed some old types
  Removed old disabled code
storage/ndb/src/mgmsrv/main.cpp:
  Removed some old types
storage/ndb/src/ndbapi/NdbBlob.cpp:
  Removed some old types
storage/ndb/src/ndbapi/NdbOperationDefine.cpp:
  Removed not used variable
storage/ndb/src/ndbapi/NdbOperationInt.cpp:
  Added required casts
storage/ndb/src/ndbapi/NdbScanOperation.cpp:
  Added required casts
storage/ndb/tools/delete_all.cpp:
  Removed some old types
storage/ndb/tools/desc.cpp:
  Removed some old types
storage/ndb/tools/drop_index.cpp:
  Removed some old types
storage/ndb/tools/drop_tab.cpp:
  Removed some old types
storage/ndb/tools/listTables.cpp:
  Removed some old types
storage/ndb/tools/ndb_config.cpp:
  Removed some old types
storage/ndb/tools/restore/consumer_restore.cpp:
  Changed some buffers to be uchar* to avoid casts with new defintion of packfrm()
storage/ndb/tools/restore/restore_main.cpp:
  Removed some old types
storage/ndb/tools/select_all.cpp:
  Removed some old types
storage/ndb/tools/select_count.cpp:
  Removed some old types
storage/ndb/tools/waiter.cpp:
  Removed some old types
strings/bchange.c:
  Changed function to use uchar * and size_t
strings/bcmp.c:
  Changed function to use uchar * and size_t
strings/bmove512.c:
  Changed function to use uchar * and size_t
strings/bmove_upp.c:
  Changed function to use uchar * and size_t
strings/ctype-big5.c:
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-bin.c:
  Changed functions to use size_t
strings/ctype-cp932.c:
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-czech.c:
  Fixed indentation
  Changed functions to use size_t
strings/ctype-euc_kr.c:
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-eucjpms.c:
  Changed functions to use size_t
  Changed character length functions to return uint
  unsigned char -> uchar
strings/ctype-gb2312.c:
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-gbk.c:
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-latin1.c:
  Changed functions to use size_t
  Changed character length functions to return uint
  unsigned char -> uchar
strings/ctype-mb.c:
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-simple.c:
  Changed functions to use size_t
  Simpler loops for caseup/casedown
  unsigned int -> uint
  unsigned char -> uchar
strings/ctype-sjis.c:
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-tis620.c:
  Changed functions to use size_t
  Changed character length functions to return uint
  unsigned char -> uchar
strings/ctype-uca.c:
  Changed functions to use size_t
  unsigned char -> uchar
strings/ctype-ucs2.c:
  Moved inclusion of stdarg.h to other includes
  usigned char -> uchar
  Changed functions to use size_t
  Changed character length functions to return uint
strings/ctype-ujis.c:
  Changed functions to use size_t
  Changed character length functions to return uint
  unsigned char -> uchar
strings/ctype-utf8.c:
  Changed functions to use size_t
  unsigned char -> uchar
  Indentation fixes
strings/ctype-win1250ch.c:
  Indentation fixes
  Changed functions to use size_t
strings/ctype.c:
  Changed functions to use size_t
strings/decimal.c:
  Changed type for memory argument to uchar *
strings/do_ctype.c:
  Indentation fixes
strings/my_strtoll10.c:
  unsigned char -> uchar
strings/my_vsnprintf.c:
  Changed functions to use size_t
strings/r_strinstr.c:
  Removed some old types
  Changed functions to use size_t
strings/str_test.c:
  Removed some old types
strings/strappend.c:
  Changed functions to use size_t
strings/strcont.c:
  Removed some old types
strings/strfill.c:
  Removed some old types
strings/strinstr.c:
  Changed functions to use size_t
strings/strlen.c:
  Changed functions to use size_t
strings/strmake.c:
  Changed functions to use size_t
strings/strnlen.c:
  Changed functions to use size_t
strings/strnmov.c:
  Changed functions to use size_t
strings/strto.c:
  unsigned char -> uchar
strings/strtod.c:
  Changed functions to use size_t
strings/strxnmov.c:
  Changed functions to use size_t
strings/xml.c:
  Changed functions to use size_t
  Indentation fixes
tests/mysql_client_test.c:
  Removed some old types
tests/thread_test.c:
  Removed some old types
vio/test-ssl.c:
  Removed some old types
vio/test-sslclient.c:
  Removed some old types
vio/test-sslserver.c:
  Removed some old types
vio/vio.c:
  Removed some old types
vio/vio_priv.h:
  Removed some old types
  Changed vio_read()/vio_write() to work with size_t
vio/viosocket.c:
  Changed vio_read()/vio_write() to work with size_t
  Indentation fixes
vio/viossl.c:
  Changed vio_read()/vio_write() to work with size_t
  Indentation fixes
vio/viosslfactories.c:
  Removed some old types
vio/viotest-ssl.c:
  Removed some old types
win/README:
  More explanations
2007-05-10 12:59:39 +03:00

1656 lines
49 KiB
C
Raw Blame History

/* Copyright (C) 2000-2006 MySQL AB
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
/* Functions to compressed records */
#include "fulltext.h"
#define IS_CHAR ((uint) 32768) /* Bit if char (not offset) in tree */
/* Some definitions to keep in sync with myisampack.c */
#define HEAD_LENGTH 32 /* Length of fixed header */
#if INT_MAX > 32767
#define BITS_SAVED 32
#define MAX_QUICK_TABLE_BITS 9 /* Because we may shift in 24 bits */
#else
#define BITS_SAVED 16
#define MAX_QUICK_TABLE_BITS 6
#endif
#define get_bit(BU) ((BU)->bits ? \
(BU)->current_byte & ((mi_bit_type) 1 << --(BU)->bits) :\
(fill_buffer(BU), (BU)->bits= BITS_SAVED-1,\
(BU)->current_byte & ((mi_bit_type) 1 << (BITS_SAVED-1))))
#define skip_to_next_byte(BU) ((BU)->bits&=~7)
#define get_bits(BU,count) (((BU)->bits >= count) ? (((BU)->current_byte >> ((BU)->bits-=count)) & mask[count]) : fill_and_get_bits(BU,count))
#define decode_bytes_test_bit(bit) \
if (low_byte & (1 << (7-bit))) \
pos++; \
if (*pos & IS_CHAR) \
{ bits-=(bit+1); break; } \
pos+= *pos
/* Size in uint16 of a Huffman tree for byte compression of 256 byte values. */
#define OFFSET_TABLE_SIZE 512
static uint read_huff_table(MI_BIT_BUFF *bit_buff,MI_DECODE_TREE *decode_tree,
uint16 **decode_table,uchar **intervall_buff,
uint16 *tmp_buff);
static void make_quick_table(uint16 *to_table,uint16 *decode_table,
uint *next_free,uint value,uint bits,
uint max_bits);
static void fill_quick_table(uint16 *table,uint bits, uint max_bits,
uint value);
static uint copy_decode_table(uint16 *to_pos,uint offset,
uint16 *decode_table);
static uint find_longest_bitstream(uint16 *table, uint16 *end);
static void (*get_unpack_function(MI_COLUMNDEF *rec))(MI_COLUMNDEF *field,
MI_BIT_BUFF *buff,
uchar *to,
uchar *end);
static void uf_zerofill_skip_zero(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_skip_zero(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_space_normal(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_space_endspace_selected(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end);
static void uf_endspace_selected(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_space_endspace(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_endspace(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_space_prespace_selected(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end);
static void uf_prespace_selected(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_space_prespace(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_prespace(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_zerofill_normal(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_constant(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_intervall(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_zero(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static void uf_blob(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end);
static void uf_varchar1(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end);
static void uf_varchar2(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end);
static void decode_bytes(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,
uchar *to,uchar *end);
static uint decode_pos(MI_BIT_BUFF *bit_buff,MI_DECODE_TREE *decode_tree);
static void init_bit_buffer(MI_BIT_BUFF *bit_buff,uchar *buffer,uint length);
static uint fill_and_get_bits(MI_BIT_BUFF *bit_buff,uint count);
static void fill_buffer(MI_BIT_BUFF *bit_buff);
static uint max_bit(uint value);
#ifdef HAVE_MMAP
static uchar *_mi_mempack_get_block_info(MI_INFO *myisam, MI_BIT_BUFF *bit_buff,
MI_BLOCK_INFO *info, uchar **rec_buff_p,
uchar *header);
#endif
static mi_bit_type mask[]=
{
0x00000000,
0x00000001, 0x00000003, 0x00000007, 0x0000000f,
0x0000001f, 0x0000003f, 0x0000007f, 0x000000ff,
0x000001ff, 0x000003ff, 0x000007ff, 0x00000fff,
0x00001fff, 0x00003fff, 0x00007fff, 0x0000ffff,
#if BITS_SAVED > 16
0x0001ffff, 0x0003ffff, 0x0007ffff, 0x000fffff,
0x001fffff, 0x003fffff, 0x007fffff, 0x00ffffff,
0x01ffffff, 0x03ffffff, 0x07ffffff, 0x0fffffff,
0x1fffffff, 0x3fffffff, 0x7fffffff, 0xffffffff,
#endif
};
/* Read all packed info, allocate memory and fix field structs */
my_bool _mi_read_pack_info(MI_INFO *info, pbool fix_keys)
{
File file;
int diff_length;
uint i,trees,huff_tree_bits,rec_reflength,length;
uint16 *decode_table,*tmp_buff;
ulong elements,intervall_length;
char *disk_cache;
uchar *intervall_buff;
uchar header[HEAD_LENGTH];
MYISAM_SHARE *share=info->s;
MI_BIT_BUFF bit_buff;
DBUG_ENTER("_mi_read_pack_info");
if (myisam_quick_table_bits < 4)
myisam_quick_table_bits=4;
else if (myisam_quick_table_bits > MAX_QUICK_TABLE_BITS)
myisam_quick_table_bits=MAX_QUICK_TABLE_BITS;
file=info->dfile;
my_errno=0;
if (my_read(file,(uchar*) header,sizeof(header),MYF(MY_NABP)))
{
if (!my_errno)
my_errno=HA_ERR_END_OF_FILE;
goto err0;
}
/* Only the first three bytes of magic number are independent of version. */
if (memcmp((uchar*) header, (uchar*) myisam_pack_file_magic, 3))
{
my_errno=HA_ERR_WRONG_IN_RECORD;
goto err0;
}
share->pack.version= header[3]; /* fourth byte of magic number */
share->pack.header_length= uint4korr(header+4);
share->min_pack_length=(uint) uint4korr(header+8);
share->max_pack_length=(uint) uint4korr(header+12);
set_if_bigger(share->base.pack_reclength,share->max_pack_length);
elements=uint4korr(header+16);
intervall_length=uint4korr(header+20);
trees=uint2korr(header+24);
share->pack.ref_length=header[26];
rec_reflength=header[27];
diff_length=(int) rec_reflength - (int) share->base.rec_reflength;
if (fix_keys)
share->rec_reflength=rec_reflength;
share->base.min_block_length=share->min_pack_length+1;
if (share->min_pack_length > 254)
share->base.min_block_length+=2;
DBUG_PRINT("info", ("fixed header length: %u", HEAD_LENGTH));
DBUG_PRINT("info", ("total header length: %lu", share->pack.header_length));
DBUG_PRINT("info", ("pack file version: %u", share->pack.version));
DBUG_PRINT("info", ("min pack length: %lu", share->min_pack_length));
DBUG_PRINT("info", ("max pack length: %lu", share->max_pack_length));
DBUG_PRINT("info", ("elements of all trees: %lu", elements));
DBUG_PRINT("info", ("distinct values bytes: %lu", intervall_length));
DBUG_PRINT("info", ("number of code trees: %u", trees));
DBUG_PRINT("info", ("bytes for record lgt: %u", share->pack.ref_length));
DBUG_PRINT("info", ("record pointer length: %u", rec_reflength));
/*
Memory segment #1:
- Decode tree heads
- Distinct column values
*/
if (!(share->decode_trees=(MI_DECODE_TREE*)
my_malloc((uint) (trees*sizeof(MI_DECODE_TREE)+
intervall_length*sizeof(uchar)),
MYF(MY_WME))))
goto err0;
intervall_buff=(uchar*) (share->decode_trees+trees);
/*
Memory segment #2:
- Decode tables
- Quick decode tables
- Temporary decode table
- Compressed data file header cache
This segment will be reallocated after construction of the tables.
*/
length=(uint) (elements*2+trees*(1 << myisam_quick_table_bits));
if (!(share->decode_tables=(uint16*)
my_malloc((length + OFFSET_TABLE_SIZE) * sizeof(uint16) +
(uint) (share->pack.header_length - sizeof(header)),
MYF(MY_WME | MY_ZEROFILL))))
goto err1;
tmp_buff=share->decode_tables+length;
disk_cache=(uchar*) (tmp_buff+OFFSET_TABLE_SIZE);
if (my_read(file,disk_cache,
(uint) (share->pack.header_length-sizeof(header)),
MYF(MY_NABP)))
goto err2;
huff_tree_bits=max_bit(trees ? trees-1 : 0);
init_bit_buffer(&bit_buff, (uchar*) disk_cache,
(uint) (share->pack.header_length-sizeof(header)));
/* Read new info for each field */
for (i=0 ; i < share->base.fields ; i++)
{
share->rec[i].base_type=(enum en_fieldtype) get_bits(&bit_buff,5);
share->rec[i].pack_type=(uint) get_bits(&bit_buff,6);
share->rec[i].space_length_bits=get_bits(&bit_buff,5);
share->rec[i].huff_tree=share->decode_trees+(uint) get_bits(&bit_buff,
huff_tree_bits);
share->rec[i].unpack=get_unpack_function(share->rec+i);
DBUG_PRINT("info", ("col: %2u type: %2u pack: %u slbits: %2u",
i, share->rec[i].base_type, share->rec[i].pack_type,
share->rec[i].space_length_bits));
}
skip_to_next_byte(&bit_buff);
/*
Construct the decoding tables from the file header. Keep track of
the used memory.
*/
decode_table=share->decode_tables;
for (i=0 ; i < trees ; i++)
if (read_huff_table(&bit_buff,share->decode_trees+i,&decode_table,
&intervall_buff,tmp_buff))
goto err3;
/* Reallocate the decoding tables to the used size. */
decode_table=(uint16*)
my_realloc((uchar*) share->decode_tables,
(uint) ((uchar*) decode_table - (uchar*) share->decode_tables),
MYF(MY_HOLD_ON_ERROR));
/* Fix the table addresses in the tree heads. */
{
long diff=PTR_BYTE_DIFF(decode_table,share->decode_tables);
share->decode_tables=decode_table;
for (i=0 ; i < trees ; i++)
share->decode_trees[i].table=ADD_TO_PTR(share->decode_trees[i].table,
diff, uint16*);
}
/* Fix record-ref-length for keys */
if (fix_keys)
{
for (i=0 ; i < share->base.keys ; i++)
{
MI_KEYDEF *keyinfo= &share->keyinfo[i];
keyinfo->keylength+= (uint16) diff_length;
keyinfo->minlength+= (uint16) diff_length;
keyinfo->maxlength+= (uint16) diff_length;
keyinfo->seg[keyinfo->flag & HA_FULLTEXT ?
FT_SEGS : keyinfo->keysegs].length= (uint16) rec_reflength;
}
if (share->ft2_keyinfo.seg)
{
MI_KEYDEF *ft2_keyinfo= &share->ft2_keyinfo;
ft2_keyinfo->keylength+= (uint16) diff_length;
ft2_keyinfo->minlength+= (uint16) diff_length;
ft2_keyinfo->maxlength+= (uint16) diff_length;
}
}
if (bit_buff.error || bit_buff.pos < bit_buff.end)
goto err3;
DBUG_RETURN(0);
err3:
my_errno=HA_ERR_WRONG_IN_RECORD;
err2:
my_free((uchar*) share->decode_tables,MYF(0));
err1:
my_free((uchar*) share->decode_trees,MYF(0));
err0:
DBUG_RETURN(1);
}
/*
Read a huff-code-table from datafile.
SYNOPSIS
read_huff_table()
bit_buff Bit buffer pointing at start of the
decoding table in the file header cache.
decode_tree Pointer to the decode tree head.
decode_table IN/OUT Address of a pointer to the next free space.
intervall_buff IN/OUT Address of a pointer to the next unused values.
tmp_buff Buffer for temporary extraction of a full
decoding table as read from bit_buff.
RETURN
0 OK.
1 Error.
*/
static uint read_huff_table(MI_BIT_BUFF *bit_buff, MI_DECODE_TREE *decode_tree,
uint16 **decode_table, uchar **intervall_buff,
uint16 *tmp_buff)
{
uint min_chr,elements,char_bits,offset_bits,size,intervall_length,table_bits,
next_free_offset;
uint16 *ptr,*end;
DBUG_ENTER("read_huff_table");
if (!get_bits(bit_buff,1))
{
/* Byte value compression. */
min_chr=get_bits(bit_buff,8);
elements=get_bits(bit_buff,9);
char_bits=get_bits(bit_buff,5);
offset_bits=get_bits(bit_buff,5);
intervall_length=0;
ptr=tmp_buff;
DBUG_PRINT("info", ("byte value compression"));
DBUG_PRINT("info", ("minimum byte value: %u", min_chr));
DBUG_PRINT("info", ("number of tree nodes: %u", elements));
DBUG_PRINT("info", ("bits for values: %u", char_bits));
DBUG_PRINT("info", ("bits for tree offsets: %u", offset_bits));
if (elements > 256)
{
DBUG_PRINT("error", ("ERROR: illegal number of tree elements: %u",
elements));
DBUG_RETURN(1);
}
}
else
{
/* Distinct column value compression. */
min_chr=0;
elements=get_bits(bit_buff,15);
intervall_length=get_bits(bit_buff,16);
char_bits=get_bits(bit_buff,5);
offset_bits=get_bits(bit_buff,5);
decode_tree->quick_table_bits=0;
ptr= *decode_table;
DBUG_PRINT("info", ("distinct column value compression"));
DBUG_PRINT("info", ("number of tree nodes: %u", elements));
DBUG_PRINT("info", ("value buffer length: %u", intervall_length));
DBUG_PRINT("info", ("bits for value index: %u", char_bits));
DBUG_PRINT("info", ("bits for tree offsets: %u", offset_bits));
}
size=elements*2-2;
DBUG_PRINT("info", ("tree size in uint16: %u", size));
DBUG_PRINT("info", ("tree size in bytes: %u",
size * (uint) sizeof(uint16)));
for (end=ptr+size ; ptr < end ; ptr++)
{
if (get_bit(bit_buff))
{
*ptr= (uint16) get_bits(bit_buff,offset_bits);
if ((ptr + *ptr >= end) || !*ptr)
{
DBUG_PRINT("error", ("ERROR: illegal pointer in decode tree"));
DBUG_RETURN(1);
}
}
else
*ptr= (uint16) (IS_CHAR + (get_bits(bit_buff,char_bits) + min_chr));
}
skip_to_next_byte(bit_buff);
decode_tree->table= *decode_table;
decode_tree->intervalls= *intervall_buff;
if (! intervall_length)
{
/* Byte value compression. ptr started from tmp_buff. */
/* Find longest Huffman code from begin to end of tree in bits. */
table_bits= find_longest_bitstream(tmp_buff, ptr);
if (table_bits >= OFFSET_TABLE_SIZE)
DBUG_RETURN(1);
if (table_bits > myisam_quick_table_bits)
table_bits=myisam_quick_table_bits;
DBUG_PRINT("info", ("table bits: %u", table_bits));
next_free_offset= (1 << table_bits);
make_quick_table(*decode_table,tmp_buff,&next_free_offset,0,table_bits,
table_bits);
(*decode_table)+= next_free_offset;
decode_tree->quick_table_bits=table_bits;
}
else
{
/* Distinct column value compression. ptr started from *decode_table */
(*decode_table)=end;
/*
get_bits() moves some bytes to a cache buffer in advance. May need
to step back.
*/
bit_buff->pos-= bit_buff->bits/8;
/* Copy the distinct column values from the buffer. */
memcpy(*intervall_buff,bit_buff->pos,(size_t) intervall_length);
(*intervall_buff)+=intervall_length;
bit_buff->pos+=intervall_length;
bit_buff->bits=0;
}
DBUG_RETURN(0);
}
/*
Make a quick_table for faster decoding.
SYNOPSIS
make_quick_table()
to_table Target quick_table and remaining decode table.
decode_table Source Huffman (sub-)tree within tmp_buff.
next_free_offset IN/OUT Next free offset from to_table.
Starts behind quick_table on the top-level.
value Huffman bits found so far.
bits Remaining bits to be collected.
max_bits Total number of bits to collect (table_bits).
DESCRIPTION
The quick table is an array of 16-bit values. There exists one value
for each possible code representable by max_bits (table_bits) bits.
In most cases table_bits is 9. So there are 512 16-bit values.
If the high-order bit (16) is set (IS_CHAR) then the array slot for
this value is a valid Huffman code for a resulting byte value.
The low-order 8 bits (1..8) are the resulting byte value.
Bits 9..14 are the length of the Huffman code for this byte value.
This means so many bits from the input stream were needed to
represent this byte value. The remaining bits belong to later
Huffman codes. This also means that for every Huffman code shorter
than table_bits there are multiple entires in the array, which
differ just in the unused bits.
If the high-order bit (16) is clear (0) then the remaining bits are
the position of the remaining Huffman decode tree segment behind the
quick table.
RETURN
void
*/
static void make_quick_table(uint16 *to_table, uint16 *decode_table,
uint *next_free_offset, uint value, uint bits,
uint max_bits)
{
DBUG_ENTER("make_quick_table");
/*
When down the table to the requested maximum, copy the rest of the
Huffman table.
*/
if (!bits--)
{
/*
Remaining left Huffman tree segment starts behind quick table.
Remaining right Huffman tree segment starts behind left segment.
*/
to_table[value]= (uint16) *next_free_offset;
/*
Re-construct the remaining Huffman tree segment at
next_free_offset in to_table.
*/
*next_free_offset= copy_decode_table(to_table, *next_free_offset,
decode_table);
DBUG_VOID_RETURN;
}
/* Descent on the left side. Left side bits are clear (0). */
if (!(*decode_table & IS_CHAR))
{
/* Not a leaf. Follow the pointer. */
make_quick_table(to_table, decode_table + *decode_table,
next_free_offset, value, bits, max_bits);
}
else
{
/*
A leaf. A Huffman code is complete. Fill the quick_table
array for all possible bit strings starting with this Huffman
code.
*/
fill_quick_table(to_table + value, bits, max_bits, (uint) *decode_table);
}
/* Descent on the right side. Right side bits are set (1). */
decode_table++;
value|= (1 << bits);
if (!(*decode_table & IS_CHAR))
{
/* Not a leaf. Follow the pointer. */
make_quick_table(to_table, decode_table + *decode_table,
next_free_offset, value, bits, max_bits);
}
else
{
/*
A leaf. A Huffman code is complete. Fill the quick_table
array for all possible bit strings starting with this Huffman
code.
*/
fill_quick_table(to_table + value, bits, max_bits, (uint) *decode_table);
}
DBUG_VOID_RETURN;
}
/*
Fill quick_table for all possible values starting with this Huffman code.
SYNOPSIS
fill_quick_table()
table Target quick_table position.
bits Unused bits from max_bits.
max_bits Total number of bits to collect (table_bits).
value The byte encoded by the found Huffman code.
DESCRIPTION
Fill the segment (all slots) of the quick_table array with the
resulting value for the found Huffman code. There are as many slots
as there are combinations representable by the unused bits.
In most cases we use 9 table bits. Assume a 3-bit Huffman code. Then
there are 6 unused bits. Hence we fill 2**6 = 64 slots with the
value.
RETURN
void
*/
static void fill_quick_table(uint16 *table, uint bits, uint max_bits,
uint value)
{
uint16 *end;
DBUG_ENTER("fill_quick_table");
/*
Bits 1..8 of value represent the decoded byte value.
Bits 9..14 become the length of the Huffman code for this byte value.
Bit 16 flags a valid code (IS_CHAR).
*/
value|= (max_bits - bits) << 8 | IS_CHAR;
for (end= table + (uint) (((uint) 1 << bits)); table < end; table++)
{
*table= (uint16) value;
}
DBUG_VOID_RETURN;
}
/*
Reconstruct a decode subtree at the target position.
SYNOPSIS
copy_decode_table()
to_pos Target quick_table and remaining decode table.
offset Next free offset from to_pos.
decode_table Source Huffman subtree within tmp_buff.
NOTE
Pointers in the decode tree are relative to the pointers position.
RETURN
next free offset from to_pos.
*/
static uint copy_decode_table(uint16 *to_pos, uint offset,
uint16 *decode_table)
{
uint prev_offset= offset;
DBUG_ENTER("copy_decode_table");
/* Descent on the left side. */
if (!(*decode_table & IS_CHAR))
{
/* Set a pointer to the next target node. */
to_pos[offset]=2;
/* Copy the left hand subtree there. */
offset=copy_decode_table(to_pos,offset+2,decode_table+ *decode_table);
}
else
{
/* Copy the byte value. */
to_pos[offset]= *decode_table;
/* Step behind this node. */
offset+=2;
}
/* Descent on the right side. */
decode_table++;
if (!(*decode_table & IS_CHAR))
{
/* Set a pointer to the next free target node. */
to_pos[prev_offset+1]=(uint16) (offset-prev_offset-1);
/* Copy the right hand subtree to the entry of that node. */
offset=copy_decode_table(to_pos,offset,decode_table+ *decode_table);
}
else
{
/* Copy the byte value. */
to_pos[prev_offset+1]= *decode_table;
}
DBUG_RETURN(offset);
}
/*
Find the length of the longest Huffman code in this table in bits.
SYNOPSIS
find_longest_bitstream()
table Code (sub-)table start.
end End of code table.
IMPLEMENTATION
Recursively follow the branch(es) of the code pair on every level of
the tree until two byte values (and no branch) are found. Add one to
each level when returning back from each recursion stage.
'end' is used for error checking only. A clean tree terminates
before reaching 'end'. Hence the exact value of 'end' is not too
important. However having it higher than necessary could lead to
misbehaviour should 'next' jump into the dirty area.
RETURN
length Length of longest Huffman code in bits.
>= OFFSET_TABLE_SIZE Error, broken tree. It does not end before 'end'.
*/
static uint find_longest_bitstream(uint16 *table, uint16 *end)
{
uint length= 1;
uint length2;
if (!(*table & IS_CHAR))
{
uint16 *next= table + *table;
if (next > end || next == table)
{
DBUG_PRINT("error", ("ERROR: illegal pointer in decode tree"));
return OFFSET_TABLE_SIZE;
}
length= find_longest_bitstream(next, end) + 1;
}
table++;
if (!(*table & IS_CHAR))
{
uint16 *next= table + *table;
if (next > end || next == table)
{
DBUG_PRINT("error", ("ERROR: illegal pointer in decode tree"));
return OFFSET_TABLE_SIZE;
}
length2= find_longest_bitstream(next, end) + 1;
length=max(length,length2);
}
return length;
}
/*
Read record from datafile.
SYNOPSIS
_mi_read_pack_record()
info A pointer to MI_INFO.
filepos File offset of the record.
buf RETURN The buffer to receive the record.
RETURN
0 on success
HA_ERR_WRONG_IN_RECORD or -1 on error
*/
int _mi_read_pack_record(MI_INFO *info, my_off_t filepos, uchar *buf)
{
MI_BLOCK_INFO block_info;
File file;
DBUG_ENTER("mi_read_pack_record");
if (filepos == HA_OFFSET_ERROR)
DBUG_RETURN(-1); /* _search() didn't find record */
file=info->dfile;
if (_mi_pack_get_block_info(info, &info->bit_buff, &block_info,
&info->rec_buff, file, filepos))
goto err;
if (my_read(file,(uchar*) info->rec_buff + block_info.offset ,
block_info.rec_len - block_info.offset, MYF(MY_NABP)))
goto panic;
info->update|= HA_STATE_AKTIV;
DBUG_RETURN(_mi_pack_rec_unpack(info, &info->bit_buff, buf,
info->rec_buff, block_info.rec_len));
panic:
my_errno=HA_ERR_WRONG_IN_RECORD;
err:
DBUG_RETURN(-1);
}
int _mi_pack_rec_unpack(register MI_INFO *info, MI_BIT_BUFF *bit_buff,
register uchar *to, uchar *from, ulong reclength)
{
uchar *end_field;
reg3 MI_COLUMNDEF *end;
MI_COLUMNDEF *current_field;
MYISAM_SHARE *share=info->s;
DBUG_ENTER("_mi_pack_rec_unpack");
init_bit_buffer(bit_buff, (uchar*) from, reclength);
for (current_field=share->rec, end=current_field+share->base.fields ;
current_field < end ;
current_field++,to=end_field)
{
end_field=to+current_field->length;
(*current_field->unpack)(current_field, bit_buff, (uchar*) to,
(uchar*) end_field);
}
if (!bit_buff->error &&
bit_buff->pos - bit_buff->bits / 8 == bit_buff->end)
DBUG_RETURN(0);
info->update&= ~HA_STATE_AKTIV;
DBUG_RETURN(my_errno=HA_ERR_WRONG_IN_RECORD);
} /* _mi_pack_rec_unpack */
/* Return function to unpack field */
static void (*get_unpack_function(MI_COLUMNDEF *rec))
(MI_COLUMNDEF *, MI_BIT_BUFF *, uchar *, uchar *)
{
switch (rec->base_type) {
case FIELD_SKIP_ZERO:
if (rec->pack_type & PACK_TYPE_ZERO_FILL)
return &uf_zerofill_skip_zero;
return &uf_skip_zero;
case FIELD_NORMAL:
if (rec->pack_type & PACK_TYPE_SPACE_FIELDS)
return &uf_space_normal;
if (rec->pack_type & PACK_TYPE_ZERO_FILL)
return &uf_zerofill_normal;
return &decode_bytes;
case FIELD_SKIP_ENDSPACE:
if (rec->pack_type & PACK_TYPE_SPACE_FIELDS)
{
if (rec->pack_type & PACK_TYPE_SELECTED)
return &uf_space_endspace_selected;
return &uf_space_endspace;
}
if (rec->pack_type & PACK_TYPE_SELECTED)
return &uf_endspace_selected;
return &uf_endspace;
case FIELD_SKIP_PRESPACE:
if (rec->pack_type & PACK_TYPE_SPACE_FIELDS)
{
if (rec->pack_type & PACK_TYPE_SELECTED)
return &uf_space_prespace_selected;
return &uf_space_prespace;
}
if (rec->pack_type & PACK_TYPE_SELECTED)
return &uf_prespace_selected;
return &uf_prespace;
case FIELD_CONSTANT:
return &uf_constant;
case FIELD_INTERVALL:
return &uf_intervall;
case FIELD_ZERO:
case FIELD_CHECK:
return &uf_zero;
case FIELD_BLOB:
return &uf_blob;
case FIELD_VARCHAR:
if (rec->length <= 256) /* 255 + 1 byte length */
return &uf_varchar1;
return &uf_varchar2;
case FIELD_LAST:
default:
return 0; /* This should never happend */
}
}
/* The different functions to unpack a field */
static void uf_zerofill_skip_zero(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end)
{
if (get_bit(bit_buff))
bzero((char*) to,(uint) (end-to));
else
{
end-=rec->space_length_bits;
decode_bytes(rec,bit_buff,to,end);
bzero((char*) end,rec->space_length_bits);
}
}
static void uf_skip_zero(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
if (get_bit(bit_buff))
bzero((char*) to,(uint) (end-to));
else
decode_bytes(rec,bit_buff,to,end);
}
static void uf_space_normal(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
if (get_bit(bit_buff))
bfill((uchar*) to,(end-to),' ');
else
decode_bytes(rec,bit_buff,to,end);
}
static void uf_space_endspace_selected(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end)
{
uint spaces;
if (get_bit(bit_buff))
bfill((uchar*) to,(end-to),' ');
else
{
if (get_bit(bit_buff))
{
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
if (to+spaces != end)
decode_bytes(rec,bit_buff,to,end-spaces);
bfill((uchar*) end-spaces,spaces,' ');
}
else
decode_bytes(rec,bit_buff,to,end);
}
}
static void uf_endspace_selected(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end)
{
uint spaces;
if (get_bit(bit_buff))
{
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
if (to+spaces != end)
decode_bytes(rec,bit_buff,to,end-spaces);
bfill((uchar*) end-spaces,spaces,' ');
}
else
decode_bytes(rec,bit_buff,to,end);
}
static void uf_space_endspace(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
uint spaces;
if (get_bit(bit_buff))
bfill((uchar*) to,(end-to),' ');
else
{
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
if (to+spaces != end)
decode_bytes(rec,bit_buff,to,end-spaces);
bfill((uchar*) end-spaces,spaces,' ');
}
}
static void uf_endspace(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
uint spaces;
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
if (to+spaces != end)
decode_bytes(rec,bit_buff,to,end-spaces);
bfill((uchar*) end-spaces,spaces,' ');
}
static void uf_space_prespace_selected(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end)
{
uint spaces;
if (get_bit(bit_buff))
bfill((uchar*) to,(end-to),' ');
else
{
if (get_bit(bit_buff))
{
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
bfill((uchar*) to,spaces,' ');
if (to+spaces != end)
decode_bytes(rec,bit_buff,to+spaces,end);
}
else
decode_bytes(rec,bit_buff,to,end);
}
}
static void uf_prespace_selected(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end)
{
uint spaces;
if (get_bit(bit_buff))
{
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
bfill((uchar*) to,spaces,' ');
if (to+spaces != end)
decode_bytes(rec,bit_buff,to+spaces,end);
}
else
decode_bytes(rec,bit_buff,to,end);
}
static void uf_space_prespace(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
uint spaces;
if (get_bit(bit_buff))
bfill((uchar*) to,(end-to),' ');
else
{
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
bfill((uchar*) to,spaces,' ');
if (to+spaces != end)
decode_bytes(rec,bit_buff,to+spaces,end);
}
}
static void uf_prespace(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
uint spaces;
if ((spaces=get_bits(bit_buff,rec->space_length_bits))+to > end)
{
bit_buff->error=1;
return;
}
bfill((uchar*) to,spaces,' ');
if (to+spaces != end)
decode_bytes(rec,bit_buff,to+spaces,end);
}
static void uf_zerofill_normal(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
end-=rec->space_length_bits;
decode_bytes(rec,bit_buff,(uchar*) to,end);
bzero((char*) end,rec->space_length_bits);
}
static void uf_constant(MI_COLUMNDEF *rec,
MI_BIT_BUFF *bit_buff __attribute__((unused)),
uchar *to,
uchar *end)
{
memcpy(to,rec->huff_tree->intervalls,(size_t) (end-to));
}
static void uf_intervall(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
reg1 uint field_length=(uint) (end-to);
memcpy(to,rec->huff_tree->intervalls+field_length*decode_pos(bit_buff,
rec->huff_tree),
(size_t) field_length);
}
/*ARGSUSED*/
static void uf_zero(MI_COLUMNDEF *rec __attribute__((unused)),
MI_BIT_BUFF *bit_buff __attribute__((unused)),
uchar *to, uchar *end)
{
bzero((char*) to,(uint) (end-to));
}
static void uf_blob(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end)
{
if (get_bit(bit_buff))
bzero((uchar*) to,(end-to));
else
{
ulong length=get_bits(bit_buff,rec->space_length_bits);
uint pack_length=(uint) (end-to)-mi_portable_sizeof_char_ptr;
if (bit_buff->blob_pos+length > bit_buff->blob_end)
{
bit_buff->error=1;
bzero((uchar*) to,(end-to));
return;
}
decode_bytes(rec,bit_buff,bit_buff->blob_pos,bit_buff->blob_pos+length);
_my_store_blob_length((uchar*) to,pack_length,length);
memcpy_fixed((char*) to+pack_length,(char*) &bit_buff->blob_pos,
sizeof(char*));
bit_buff->blob_pos+=length;
}
}
static void uf_varchar1(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end __attribute__((unused)))
{
if (get_bit(bit_buff))
to[0]= 0; /* Zero lengths */
else
{
ulong length=get_bits(bit_buff,rec->space_length_bits);
*to= (uchar) length;
decode_bytes(rec,bit_buff,to+1,to+1+length);
}
}
static void uf_varchar2(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff,
uchar *to, uchar *end __attribute__((unused)))
{
if (get_bit(bit_buff))
to[0]=to[1]=0; /* Zero lengths */
else
{
ulong length=get_bits(bit_buff,rec->space_length_bits);
int2store(to,length);
decode_bytes(rec,bit_buff,to+2,to+2+length);
}
}
/* Functions to decode of buffer of bits */
#if BITS_SAVED == 64
static void decode_bytes(MI_COLUMNDEF *rec,MI_BIT_BUFF *bit_buff,uchar *to,
uchar *end)
{
reg1 uint bits,low_byte;
reg3 uint16 *pos;
reg4 uint table_bits,table_and;
MI_DECODE_TREE *decode_tree;
decode_tree=rec->decode_tree;
bits=bit_buff->bits; /* Save in reg for quicker access */
table_bits=decode_tree->quick_table_bits;
table_and= (1 << table_bits)-1;
do
{
if (bits <= 32)
{
if (bit_buff->pos > bit_buff->end+4)
{
bit_buff->error=1;
return; /* Can't be right */
}
bit_buff->current_byte= (bit_buff->current_byte << 32) +
((((uint) bit_buff->pos[3])) +
(((uint) bit_buff->pos[2]) << 8) +
(((uint) bit_buff->pos[1]) << 16) +
(((uint) bit_buff->pos[0]) << 24));
bit_buff->pos+=4;
bits+=32;
}
/*
First use info in quick_table.
The quick table is an array of 16-bit values. There exists one
value for each possible code representable by table_bits bits.
In most cases table_bits is 9. So there are 512 16-bit values.
If the high-order bit (16) is set (IS_CHAR) then the array slot
for this value is a valid Huffman code for a resulting byte value.
The low-order 8 bits (1..8) are the resulting byte value.
Bits 9..14 are the length of the Huffman code for this byte value.
This means so many bits from the input stream were needed to
represent this byte value. The remaining bits belong to later
Huffman codes. This also means that for every Huffman code shorter
than table_bits there are multiple entires in the array, which
differ just in the unused bits.
If the high-order bit (16) is clear (0) then the remaining bits are
the position of the remaining Huffman decode tree segment behind the
quick table.
*/
low_byte=(uint) (bit_buff->current_byte >> (bits - table_bits)) & table_and;
low_byte=decode_tree->table[low_byte];
if (low_byte & IS_CHAR)
{
/*
All Huffman codes of less or equal table_bits length are in the
quick table. This is one of them.
*/
*to++ = (low_byte & 255); /* Found char in quick table */
bits-= ((low_byte >> 8) & 31); /* Remove bits used */
}
else
{ /* Map through rest of decode-table */
/* This means that the Huffman code must be longer than table_bits. */
pos=decode_tree->table+low_byte;
bits-=table_bits;
/* NOTE: decode_bytes_test_bit() is a macro wich contains a break !!! */
for (;;)
{
low_byte=(uint) (bit_buff->current_byte >> (bits-8));
decode_bytes_test_bit(0);
decode_bytes_test_bit(1);
decode_bytes_test_bit(2);
decode_bytes_test_bit(3);
decode_bytes_test_bit(4);
decode_bytes_test_bit(5);
decode_bytes_test_bit(6);
decode_bytes_test_bit(7);
bits-=8;
}
*to++ = *pos;
}
} while (to != end);
bit_buff->bits=bits;
return;
}
#else
static void decode_bytes(MI_COLUMNDEF *rec, MI_BIT_BUFF *bit_buff, uchar *to,
uchar *end)
{
reg1 uint bits,low_byte;
reg3 uint16 *pos;
reg4 uint table_bits,table_and;
MI_DECODE_TREE *decode_tree;
decode_tree=rec->huff_tree;
bits=bit_buff->bits; /* Save in reg for quicker access */
table_bits=decode_tree->quick_table_bits;
table_and= (1 << table_bits)-1;
do
{
if (bits < table_bits)
{
if (bit_buff->pos > bit_buff->end+1)
{
bit_buff->error=1;
return; /* Can't be right */
}
#if BITS_SAVED == 32
bit_buff->current_byte= (bit_buff->current_byte << 24) +
(((uint) ((uchar) bit_buff->pos[2]))) +
(((uint) ((uchar) bit_buff->pos[1])) << 8) +
(((uint) ((uchar) bit_buff->pos[0])) << 16);
bit_buff->pos+=3;
bits+=24;
#else
if (bits) /* We must have at leasts 9 bits */
{
bit_buff->current_byte= (bit_buff->current_byte << 8) +
(uint) ((uchar) bit_buff->pos[0]);
bit_buff->pos++;
bits+=8;
}
else
{
bit_buff->current_byte= ((uint) ((uchar) bit_buff->pos[0]) << 8) +
((uint) ((uchar) bit_buff->pos[1]));
bit_buff->pos+=2;
bits+=16;
}
#endif
}
/* First use info in quick_table */
low_byte=(bit_buff->current_byte >> (bits - table_bits)) & table_and;
low_byte=decode_tree->table[low_byte];
if (low_byte & IS_CHAR)
{
*to++ = (low_byte & 255); /* Found char in quick table */
bits-= ((low_byte >> 8) & 31); /* Remove bits used */
}
else
{ /* Map through rest of decode-table */
pos=decode_tree->table+low_byte;
bits-=table_bits;
for (;;)
{
if (bits < 8)
{ /* We don't need to check end */
#if BITS_SAVED == 32
bit_buff->current_byte= (bit_buff->current_byte << 24) +
(((uint) ((uchar) bit_buff->pos[2]))) +
(((uint) ((uchar) bit_buff->pos[1])) << 8) +
(((uint) ((uchar) bit_buff->pos[0])) << 16);
bit_buff->pos+=3;
bits+=24;
#else
bit_buff->current_byte= (bit_buff->current_byte << 8) +
(uint) ((uchar) bit_buff->pos[0]);
bit_buff->pos+=1;
bits+=8;
#endif
}
low_byte=(uint) (bit_buff->current_byte >> (bits-8));
decode_bytes_test_bit(0);
decode_bytes_test_bit(1);
decode_bytes_test_bit(2);
decode_bytes_test_bit(3);
decode_bytes_test_bit(4);
decode_bytes_test_bit(5);
decode_bytes_test_bit(6);
decode_bytes_test_bit(7);
bits-=8;
}
*to++ = (uchar) *pos;
}
} while (to != end);
bit_buff->bits=bits;
return;
}
#endif /* BIT_SAVED == 64 */
static uint decode_pos(MI_BIT_BUFF *bit_buff, MI_DECODE_TREE *decode_tree)
{
uint16 *pos=decode_tree->table;
for (;;)
{
if (get_bit(bit_buff))
pos++;
if (*pos & IS_CHAR)
return (uint) (*pos & ~IS_CHAR);
pos+= *pos;
}
}
int _mi_read_rnd_pack_record(MI_INFO *info, uchar *buf,
register my_off_t filepos,
my_bool skip_deleted_blocks)
{
uint b_type;
MI_BLOCK_INFO block_info;
MYISAM_SHARE *share=info->s;
DBUG_ENTER("_mi_read_rnd_pack_record");
if (filepos >= info->state->data_file_length)
{
my_errno= HA_ERR_END_OF_FILE;
goto err;
}
if (info->opt_flag & READ_CACHE_USED)
{
if (_mi_read_cache(&info->rec_cache, (uchar*) block_info.header,
filepos, share->pack.ref_length,
skip_deleted_blocks ? READING_NEXT : 0))
goto err;
b_type=_mi_pack_get_block_info(info, &info->bit_buff, &block_info,
&info->rec_buff, -1, filepos);
}
else
b_type=_mi_pack_get_block_info(info, &info->bit_buff, &block_info,
&info->rec_buff, info->dfile, filepos);
if (b_type)
goto err; /* Error code is already set */
#ifndef DBUG_OFF
if (block_info.rec_len > share->max_pack_length)
{
my_errno=HA_ERR_WRONG_IN_RECORD;
goto err;
}
#endif
if (info->opt_flag & READ_CACHE_USED)
{
if (_mi_read_cache(&info->rec_cache, (uchar*) info->rec_buff,
block_info.filepos, block_info.rec_len,
skip_deleted_blocks ? READING_NEXT : 0))
goto err;
}
else
{
if (my_read(info->dfile,(uchar*) info->rec_buff + block_info.offset,
block_info.rec_len-block_info.offset,
MYF(MY_NABP)))
goto err;
}
info->packed_length=block_info.rec_len;
info->lastpos=filepos;
info->nextpos=block_info.filepos+block_info.rec_len;
info->update|= HA_STATE_AKTIV | HA_STATE_KEY_CHANGED;
DBUG_RETURN (_mi_pack_rec_unpack(info, &info->bit_buff, buf,
info->rec_buff, block_info.rec_len));
err:
DBUG_RETURN(my_errno);
}
/* Read and process header from a huff-record-file */
uint _mi_pack_get_block_info(MI_INFO *myisam, MI_BIT_BUFF *bit_buff,
MI_BLOCK_INFO *info, uchar **rec_buff_p,
File file, my_off_t filepos)
{
uchar *header=info->header;
uint head_length,ref_length;
LINT_INIT(ref_length);
if (file >= 0)
{
ref_length=myisam->s->pack.ref_length;
/*
We can't use my_pread() here because mi_read_rnd_pack_record assumes
position is ok
*/
VOID(my_seek(file,filepos,MY_SEEK_SET,MYF(0)));
if (my_read(file,(char*) header,ref_length,MYF(MY_NABP)))
return BLOCK_FATAL_ERROR;
DBUG_DUMP("header",(uchar*) header,ref_length);
}
head_length= read_pack_length((uint) myisam->s->pack.version, header,
&info->rec_len);
if (myisam->s->base.blobs)
{
head_length+= read_pack_length((uint) myisam->s->pack.version,
header + head_length, &info->blob_len);
/*
Ensure that the record buffer is big enough for the compressed
record plus all expanded blobs. [We do not have an extra buffer
for the resulting blobs. Sigh.]
*/
if (!(mi_alloc_rec_buff(myisam,info->rec_len + info->blob_len,
rec_buff_p)))
return BLOCK_FATAL_ERROR; /* not enough memory */
bit_buff->blob_pos= (uchar*) *rec_buff_p + info->rec_len;
bit_buff->blob_end= bit_buff->blob_pos + info->blob_len;
myisam->blob_length=info->blob_len;
}
info->filepos=filepos+head_length;
if (file > 0)
{
info->offset=min(info->rec_len, ref_length - head_length);
memcpy(*rec_buff_p, header + head_length, info->offset);
}
return 0;
}
/* rutines for bit buffer */
/* Note buffer must be 6 byte bigger than longest row */
static void init_bit_buffer(MI_BIT_BUFF *bit_buff, uchar *buffer, uint length)
{
bit_buff->pos=buffer;
bit_buff->end=buffer+length;
bit_buff->bits=bit_buff->error=0;
bit_buff->current_byte=0; /* Avoid purify errors */
}
static uint fill_and_get_bits(MI_BIT_BUFF *bit_buff, uint count)
{
uint tmp;
count-=bit_buff->bits;
tmp=(bit_buff->current_byte & mask[bit_buff->bits]) << count;
fill_buffer(bit_buff);
bit_buff->bits=BITS_SAVED - count;
return tmp+(bit_buff->current_byte >> (BITS_SAVED - count));
}
/* Fill in empty bit_buff->current_byte from buffer */
/* Sets bit_buff->error if buffer is exhausted */
static void fill_buffer(MI_BIT_BUFF *bit_buff)
{
if (bit_buff->pos >= bit_buff->end)
{
bit_buff->error= 1;
bit_buff->current_byte=0;
return;
}
#if BITS_SAVED == 64
bit_buff->current_byte= ((((uint) ((uchar) bit_buff->pos[7]))) +
(((uint) ((uchar) bit_buff->pos[6])) << 8) +
(((uint) ((uchar) bit_buff->pos[5])) << 16) +
(((uint) ((uchar) bit_buff->pos[4])) << 24) +
((ulonglong)
((((uint) ((uchar) bit_buff->pos[3]))) +
(((uint) ((uchar) bit_buff->pos[2])) << 8) +
(((uint) ((uchar) bit_buff->pos[1])) << 16) +
(((uint) ((uchar) bit_buff->pos[0])) << 24)) << 32));
bit_buff->pos+=8;
#else
#if BITS_SAVED == 32
bit_buff->current_byte= (((uint) ((uchar) bit_buff->pos[3])) +
(((uint) ((uchar) bit_buff->pos[2])) << 8) +
(((uint) ((uchar) bit_buff->pos[1])) << 16) +
(((uint) ((uchar) bit_buff->pos[0])) << 24));
bit_buff->pos+=4;
#else
bit_buff->current_byte= (uint) (((uint) ((uchar) bit_buff->pos[1]))+
(((uint) ((uchar) bit_buff->pos[0])) << 8));
bit_buff->pos+=2;
#endif
#endif
}
/* Get number of bits neaded to represent value */
static uint max_bit(register uint value)
{
reg2 uint power=1;
while ((value>>=1))
power++;
return (power);
}
/*****************************************************************************
Some redefined functions to handle files when we are using memmap
*****************************************************************************/
#ifdef HAVE_SYS_MMAN_H
#include <sys/mman.h>
#endif
#ifdef HAVE_MMAP
static int _mi_read_mempack_record(MI_INFO *info,my_off_t filepos,uchar *buf);
static int _mi_read_rnd_mempack_record(MI_INFO*, uchar *,my_off_t, my_bool);
my_bool _mi_memmap_file(MI_INFO *info)
{
MYISAM_SHARE *share=info->s;
DBUG_ENTER("mi_memmap_file");
if (!info->s->file_map)
{
if (my_seek(info->dfile,0L,MY_SEEK_END,MYF(0)) <
share->state.state.data_file_length+MEMMAP_EXTRA_MARGIN)
{
DBUG_PRINT("warning",("File isn't extended for memmap"));
DBUG_RETURN(0);
}
if (mi_dynmap_file(info, share->state.state.data_file_length))
DBUG_RETURN(0);
}
info->opt_flag|= MEMMAP_USED;
info->read_record= share->read_record= _mi_read_mempack_record;
share->read_rnd= _mi_read_rnd_mempack_record;
DBUG_RETURN(1);
}
void _mi_unmap_file(MI_INFO *info)
{
VOID(my_munmap(info->s->file_map,
(size_t) info->s->mmaped_length + MEMMAP_EXTRA_MARGIN));
}
static uchar *_mi_mempack_get_block_info(MI_INFO *myisam, MI_BIT_BUFF *bit_buff,
MI_BLOCK_INFO *info, uchar **rec_buff_p,
uchar *header)
{
header+= read_pack_length((uint) myisam->s->pack.version, header,
&info->rec_len);
if (myisam->s->base.blobs)
{
header+= read_pack_length((uint) myisam->s->pack.version, header,
&info->blob_len);
/* mi_alloc_rec_buff sets my_errno on error */
if (!(mi_alloc_rec_buff(myisam, info->blob_len,
rec_buff_p)))
return 0; /* not enough memory */
bit_buff->blob_pos= (uchar*) *rec_buff_p;
bit_buff->blob_end= (uchar*) *rec_buff_p + info->blob_len;
}
return header;
}
static int _mi_read_mempack_record(MI_INFO *info, my_off_t filepos, uchar *buf)
{
MI_BLOCK_INFO block_info;
MYISAM_SHARE *share=info->s;
uchar *pos;
DBUG_ENTER("mi_read_mempack_record");
if (filepos == HA_OFFSET_ERROR)
DBUG_RETURN(-1); /* _search() didn't find record */
if (!(pos= (uchar*) _mi_mempack_get_block_info(info, &info->bit_buff,
&block_info, &info->rec_buff,
(uchar*) share->file_map+
filepos)))
DBUG_RETURN(-1);
DBUG_RETURN(_mi_pack_rec_unpack(info, &info->bit_buff, buf,
pos, block_info.rec_len));
}
/*ARGSUSED*/
static int _mi_read_rnd_mempack_record(MI_INFO *info, uchar *buf,
register my_off_t filepos,
my_bool skip_deleted_blocks
__attribute__((unused)))
{
MI_BLOCK_INFO block_info;
MYISAM_SHARE *share=info->s;
uchar *pos,*start;
DBUG_ENTER("_mi_read_rnd_mempack_record");
if (filepos >= share->state.state.data_file_length)
{
my_errno=HA_ERR_END_OF_FILE;
goto err;
}
if (!(pos= (uchar*) _mi_mempack_get_block_info(info, &info->bit_buff,
&block_info, &info->rec_buff,
(uchar*)
(start=share->file_map+
filepos))))
goto err;
#ifndef DBUG_OFF
if (block_info.rec_len > info->s->max_pack_length)
{
my_errno=HA_ERR_WRONG_IN_RECORD;
goto err;
}
#endif
info->packed_length=block_info.rec_len;
info->lastpos=filepos;
info->nextpos=filepos+(uint) (pos-start)+block_info.rec_len;
info->update|= HA_STATE_AKTIV | HA_STATE_KEY_CHANGED;
DBUG_RETURN (_mi_pack_rec_unpack(info, &info->bit_buff, buf,
pos, block_info.rec_len));
err:
DBUG_RETURN(my_errno);
}
#endif /* HAVE_MMAP */
/* Save length of row */
uint save_pack_length(uint version, uchar *block_buff, ulong length)
{
if (length < 254)
{
*(uchar*) block_buff= (uchar) length;
return 1;
}
if (length <= 65535)
{
*(uchar*) block_buff=254;
int2store(block_buff+1,(uint) length);
return 3;
}
*(uchar*) block_buff=255;
if (version == 1) /* old format */
{
DBUG_ASSERT(length <= 0xFFFFFF);
int3store(block_buff + 1, (ulong) length);
return 4;
}
else
{
int4store(block_buff + 1, (ulong) length);
return 5;
}
}
uint read_pack_length(uint version, const uchar *buf, ulong *length)
{
if (buf[0] < 254)
{
*length= buf[0];
return 1;
}
else if (buf[0] == 254)
{
*length= uint2korr(buf + 1);
return 3;
}
if (version == 1) /* old format */
{
*length= uint3korr(buf + 1);
return 4;
}
else
{
*length= uint4korr(buf + 1);
return 5;
}
}
uint calc_pack_length(uint version, ulong length)
{
return (length < 254) ? 1 : (length < 65536) ? 3 : (version == 1) ? 4 : 5;
}
Reference in a new issue View git blame Copy permalink