mariadb/strings
Alexander Barkov 1094ffd572 WL#2649 Number-to-string conversions
added:
  include/ctype_numconv.inc
  mysql-test/include/ctype_numconv.inc
  mysql-test/r/ctype_binary.result
  mysql-test/t/ctype_binary.test
  Adding tests

modified:

  mysql-test/r/bigint.result
  mysql-test/r/case.result
  mysql-test/r/create.result
  mysql-test/r/ctype_cp1251.result
  mysql-test/r/ctype_latin1.result
  mysql-test/r/ctype_ucs.result
  mysql-test/r/func_gconcat.result
  mysql-test/r/func_str.result
  mysql-test/r/metadata.result
  mysql-test/r/ps_1general.result
  mysql-test/r/ps_2myisam.result
  mysql-test/r/ps_3innodb.result
  mysql-test/r/ps_4heap.result
  mysql-test/r/ps_5merge.result
  mysql-test/r/show_check.result
  mysql-test/r/type_datetime.result
  mysql-test/r/type_ranges.result
  mysql-test/r/union.result
  mysql-test/suite/ndb/r/ps_7ndb.result
  mysql-test/t/ctype_cp1251.test
  mysql-test/t/ctype_latin1.test
  mysql-test/t/ctype_ucs.test
  mysql-test/t/func_str.test
    Fixing tests


  @ sql/field.cc
     - Return str result using my_charset_numeric.
     - Using real multi-byte aware str_to_XXX functions
       to handle tricky charset values propely (e.g. UCS2)
  @ sql/field.h
     - Changing derivation of non-string field types to DERIVATION_NUMERIC.
     - Changing binary() for numeric/datetime fields to always
     return TRUE even if charset is not my_charset_bin. We need
     this to keep ha_base_keytype() return HA_KEYTYPE_BINARY.
     - Adding BINARY_FLAG into some fields, because it's not
     being set automatically anymore with
     "my_charset_bin to my_charset_numeric" change.
    - Changing derivation for numeric/datetime datatypes to a weaker
      value, to make "SELECT concat('string', field)" use character
      set of the string literal for the result of the function.
  @ sql/item.cc
     - Implementing generic val_str_ascii().
     - Using max_char_length() instead of direct read of max_length
       to make "tricky" charsets like UCS2 work.
       NOTE: in the future we'll possibly remove all direct reads of max_length
     - Fixing Item_num::safe_charset_converter().
       Previously it alligned binary string to
       character string (for example by adding leading 0x00
       when doing binary->UCS2 conversion). Now it just
       converts from my_charset_numbner to "tocs".
     - Using val_str_ascii() in Item::get_time() to make UCS2 arguments work.
     - Other misc changes
  @ sql/item.h
     - Changing MY_COLL_CMP_CONV and MY_COLL_ALLOW_CONV to
       bit operations instead of hard-coded bit masks.
     - Addding new method DTCollation.set_numeric().
     - Adding new methods to Item.
     - Adding helper functions to make code look nicer:
       agg_item_charsets_for_string_result()
       agg_item_charsets_for_comparison()
     - Changing charset for Item_num-derived items
       from my_charset_bin to my_charset_numeric
       (which is an alias for latin1).
  @ sql/item_cmpfunc.cc
     - Using new helper functions
     - Other misc changes
  @ sql/item_cmpfunc.h
     - Fixing strcmp() to return max_length=2.
       Previously it returned 1, which was wrong,
       because it did not fit '-1'.
  @ sql/item_func.cc
     - Using new helper functions
     - Other minor changes
  @ sql/item_func.h
     - Removing unused functions
     - Adding helper functions
       agg_arg_charsets_for_string_result()
       agg_arg_charsets_for_comparison()
     - Adding set_numeric() into constructors of numeric items.
     - Using fix_length_and_charset() and fix_char_length()
       instead of direct write to max_length.
  @ sql/item_geofunc.cc
     - Changing class for Item_func_geometry_type and
       Item_func_as_wkt from Item_str_func to
       Item_str_ascii_func, to make them return UCS2 result
       properly (when character_set_connection=ucs2).
  @ sql/item_geofunc.h
     - Changing class for Item_func_geometry_type and
       Item_func_as_wkt from Item_str_func to
       Item_str_ascii_func, to make them return UCS2 result
       properly (when @@character_set_connection=ucs2).
  @ sql/item_strfunc.cc
     - Implementing Item_str_func::val_str().
     - Renaming val_str to val_str_ascii for some items,
       to make them work with UCS2 properly.
     - Using new helper functions
     - All single-argument functions that expect string
       result now call this method:
       agg_arg_charsets_for_string_result(collation, args, 1);
       This enables character set conversion to @@character_set_connection
       in case of pure numeric input.
  @ sql/item_strfunc.h
     - Introducing Item_str_ascii_func - for functions
       which return pure ASCII data, for performance purposes,
       as well as for the cases when the old implementation
       of val_str() was heavily 8-bit oriented and implementing
       a UCS2-aware version is tricky.
  @ sql/item_sum.cc
     - Using new helper functions.
  @ sql/item_timefunc.cc
     - Using my_charset_numeric instead of my_charset_bin.
     - Using fix_char_length(), fix_length_and_charset()
       and fix_length_and_charset_datetime()
       instead of direct write to max_length.
     - Using tricky-charset aware function str_to_time_with_warn()
  @ sql/item_timefunc.h
     - Using new helper functions for charset and length initialization.
     - Changing base class for Item_func_get_format() to make
       it return UCS2 properly (when character_set_connection=ucs2).
  @ sql/item_xmlfunc.cc
     - Using new helper function
  @ sql/my_decimal.cc
     - Adding a new DECIMAL to CHAR converter
       with real multibyte support (e.g. UCS2)

  @ sql/mysql_priv.h
     - Introducing a new derivation level for numeric/datetime data types.
     - Adding macros for my_charset_numeric and MY_REPERTOIRE_NUMERIC.
     - Adding prototypes for str_set_decimal()
     - Adding prototypes for character-set aware str_to_xxx() functions.
  @ sql/protocol.cc
     - Changing charsetnr to "binary" client-side metadata for
       numeric/datetime data types.
  @ sql/time.cc
     - Adding to_ascii() helper function, to convert a string
       in any character set to ascii representation. In the
       future can be extended to understand digits written
       in various non-Latin word scripts.
     - Adding real multy-byte character set aware versions for str_to_XXXX,
       to make these these type of queries work correct:
         INSERT INTO t1 SET datetime_column=ucs2_expression;
   @  strings/ctype-ucs2.c
     - endptr was not calculated correctly. INSERTing of UCS2
       values into numeric columns returned warnings about
       truncated wrong data.
2010-02-11 08:17:25 +04:00
..
bchange.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
bcmp.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
bcopy-duff.c Many files: 2006-12-23 20:17:15 +01:00
bfill.c Many files: 2006-12-23 20:17:15 +01:00
bmove.c Many files: 2006-12-23 20:17:15 +01:00
bmove512.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
bmove_upp-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
bmove_upp.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
bzero.c Many files: 2006-12-23 20:17:15 +01:00
ChangeLog PART I: Keys support for MyMERGE tables 2000-08-14 15:27:19 +04:00
CHARSET_INFO.txt Updating charset doc files. 2008-05-28 15:03:47 +05:00
CMakeLists.txt Backport of WL #2934: Make/find library for doing float/double 2009-12-22 19:23:13 +03:00
conf_to_src.c Backporting WL#4164 Two-byte collation IDs 2009-10-15 15:17:32 +05:00
ctype-big5.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-bin.c Bug #14637: trim trailing spaces processes data only byte wise 2009-11-11 17:03:02 +01:00
ctype-cp932.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-czech.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
ctype-euc_kr.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-eucjpms.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-extra.c Backporting WL#3759 Optimize identifier conversion in client-server protocol 2009-09-30 10:09:28 +05:00
ctype-gb2312.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-gbk.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-latin1.c Bug #14637: trim trailing spaces processes data only byte wise 2009-11-11 17:03:02 +01:00
ctype-mb.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-simple.c Bug #14637: trim trailing spaces processes data only byte wise 2009-11-11 17:03:02 +01:00
ctype-sjis.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-tis620.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
ctype-uca.c Auto-merge from mysql-next-mr. 2009-11-12 15:13:43 +03:00
ctype-ucs2.c WL#2649 Number-to-string conversions 2010-02-11 08:17:25 +04:00
ctype-ujis.c WL#4583 Case conversion in Asian character sets 2010-01-14 15:17:57 +04:00
ctype-utf8.c After-fix for back-port of WL#3759. 2009-10-01 12:22:31 +05:00
ctype-win1250ch.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
ctype.c Backporting Bug#37129 LDML lacks <i> rule 2009-11-09 13:45:40 +04:00
decimal.c Backport of WL #2934: Make/find library for doing float/double 2009-12-22 19:23:13 +03:00
do_ctype.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
dtoa.c Backport of WL #2934: Make/find library for doing float/double 2009-12-22 19:23:13 +03:00
dump_map.c my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
int2str.c BUG#31799: Scrambled number output due to integer overflow 2007-10-31 10:34:26 +01:00
is_prefix.c Many files: 2006-12-23 20:17:15 +01:00
latin2.def Import changeset 2000-07-31 21:29:14 +02:00
llstr.c Bug#31177: Server variables can't be set to their current values 2007-11-30 06:32:04 +01:00
longlong2str-x86.s Many files: 2006-12-23 20:17:15 +01:00
longlong2str.c WL#4738 streamline/simplify @@variable creation process 2009-12-22 10:35:56 +01:00
longlong2str_asm.c Merge backport of WL#3771 with mysql-next-mr. 2010-01-19 17:30:55 +04:00
macros.asm my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
make-ccc Import changeset 2000-07-31 21:29:14 +02:00
Makefile.am Manual merge from mysql-trunk-merge. 2010-01-19 19:36:14 +03:00
memcmp.c Many files: 2006-12-23 20:17:15 +01:00
memcpy.c Many files: 2006-12-23 20:17:15 +01:00
memset.c Many files: 2006-12-23 20:17:15 +01:00
my_strchr.c Fixed compiler warnings 2007-02-23 13:13:55 +02:00
my_strtoll10-x86.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
my_strtoll10.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
my_vsnprintf.c Manual merge of WL#4738 from mysql-next-mr: 2009-12-28 15:54:16 +03:00
ptr_cmp.asm my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
r_strinstr.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
README Import changeset 2000-07-31 21:29:14 +02:00
str2int.c Many files: 2006-12-23 20:17:15 +01:00
str_alloc.c Many files: 2006-12-23 20:17:15 +01:00
str_test.c Backport of: 2009-11-24 16:54:59 +03:00
strappend-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strappend.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
strcat.c Many files: 2006-12-23 20:17:15 +01:00
strcend.c Many files: 2006-12-23 20:17:15 +01:00
strchr.c Many files: 2006-12-23 20:17:15 +01:00
strcmp.c Many files: 2006-12-23 20:17:15 +01:00
strcont.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
strend-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strend.c Many files: 2006-12-23 20:17:15 +01:00
strfill.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
string.doc Import changeset 2000-07-31 21:29:14 +02:00
strings-not-used.h WL#5016: Fix header file include guards 2009-09-23 23:32:31 +02:00
strings-x86.s Many files: 2006-12-23 20:17:15 +01:00
strings.asm my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strinstr-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strinstr.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
strlen.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
strmake-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strmake.c Bug #46042: backported the fix for the valgrind warning from 5.1 2009-07-16 16:13:26 +03:00
strmov-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strmov.c Bug #48866: mysql.test fails under Fedora 12 2009-12-16 19:31:19 +02:00
strnlen.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
strnmov-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strnmov.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
strrchr.c Many files: 2006-12-23 20:17:15 +01:00
strstr-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strstr.c Many files: 2006-12-23 20:17:15 +01:00
strto.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
strtol.c Many files: 2006-12-23 20:17:15 +01:00
strtoll.c Many files: 2006-12-23 20:17:15 +01:00
strtoul.c Many files: 2006-12-23 20:17:15 +01:00
strtoull.c Many files: 2006-12-23 20:17:15 +01:00
strxmov-sparc.s my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strxmov.asm my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
strxmov.c Many files: 2006-12-23 20:17:15 +01:00
strxnmov.c WL#3817: Simplify string / memory area types and make things more consistent (first part) 2007-05-10 12:59:39 +03:00
t_ctype.h Many files: 2006-12-23 20:17:15 +01:00
uca-dump.c my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
uctypedump.c Many files: 2006-12-31 02:29:11 +01:00
udiv.c Many files: 2006-12-23 20:17:15 +01:00
utr11-dump.c my_strtoll10-x86.s: 2006-12-31 01:02:27 +01:00
xml.c Bug#29125 Windows Server X64: so many compiler warnings 2009-02-10 17:47:54 -05:00

File   : README
Author : Richard A. O'Keefe.
Updated: 30 April 1984
Purpose: Explain the new strings package.

    The UNIX string libraries (described in the string(3) manual page)
differ from UNIX to UNIX (e.g. strtok is not in V7 or 4.1bsd).  Worse,
the sources are not in the public domain, so that if there is a string
routine which is nearly what you want but not quite you can't  take  a
copy  and  modify it.  And of course C programmers on non-UNIX systems
are at the mercy of their supplier.

    This package was designed to let me do reasonable things with  C's
strings  whatever UNIX (V7, PaNiX, UX63, 4.1bsd) I happen to be using.
Everything in the System III manual is here and does just what the  S3
manual  says  it does.  There are also lots of new goodies.  I'm sorry
about the names, but the routines do have to work  on  asphyxiated-at-
birth  systems  which  truncate identifiers.  The convention is that a
routine is called
 str [n] [c] <operation>
If there is an "n", it means that the function takes an (int) "length"
argument, which bounds the number of characters to be moved or  looked
at.  If the function has a "set" argument, a "c" in the name indicates
that  the complement of the set is used.  Functions or variables whose
names start with _ are support routines which aren't really meant  for
general  use.  I don't know what the "p" is doing in "strpbrk", but it
is there in the S3 manual so it's here too.  "istrtok" does not follow
this rule, but with 7 letters what can you do?

    I have included new versions of atoi(3) and atol(3) as well.  They
use a new primitive str2int, which takes a pair of bounds and a radix,
and does much more thorough checking than the normal atoi and atol do.
The result returned by atoi & atol is valid if and only if errno == 0.
There is also an output conversion routine int2str, with itoa and ltoa
as interface macros.  Only after writing int2str did I notice that the
str2int routine has no provision for unsigned numbers.  On reflection,
I don't greatly care.   I'm afraid that int2str may depend on your "C"
compiler in unexpected ways.  Do check the code with -S.

    Several of these routines have "asm" inclusions conditional on the
VaxAsm option.  These insertions can make the routines which have them
quite a bit faster, but there is a snag.  The VAX architects, for some
reason best known to themselves and their therapists, decided that all
"strings" were shorter than 2^16 bytes.  Even when the length operands
are in 32-bit registers, only 16 bits count.  So the "asm" versions do
not work for long strings.  If you can guarantee that all your strings
will be short, define VaxAsm in the makefile, but in general, and when
using other machines, do not define it.

    To use this library, you need the "strings.a" library file and the
"strings.h" and "ctypes.h" header files.  The other header  files  are
for compiling the library itself, though if you are hacking extensions
you  may  find  them useful.  General users really shouldn't see them.
I've defined a few macros I find useful in "strings.h"; if you have no
need for "index", "rindex", "streql", and "beql", just edit them  out.
On the 4.1bsd system I am using declaring all these functions 'extern'
does not mean that they will all be loaded; but only the ones you use.
When using lesser systems you may find it necessary to break strings.h
up, or you could get by with just adding "extern" declarations for the
functions you want as you need them.  Many of these functions have the
same names as functions in the "standard C library", by design as this
is a replacement/reimplementation of part of that library.  So you may
have to talk the loader into loading this library first.   Again, I've
found no problems on 4.1bsd.

    You may wonder at my failure to provide manual pages for this code.
For the things in V7, 4.?, or SIII, you should be able to use whichever
manual page came with that system,  and anything I might write would be
so like it as to raise suspicions of violating AT&T copyrights.  In the
sources you will find comments which provide far more documentation for
these routines than AT&T ever provided for their strings stuff,  I just
don't happen to have put it in nroff -man form.   Had I done so, the .3
files would have outbulked the .c files!

    These files are in the public domain.  This includes getopt.c, which
is the work of Henry Spencer, University of Toronto Zoology, who says of
it "None of this software is derived from Bell software. I had no access
to the source for Bell's versions at the time I wrote it.  This software
is hereby explicitly placed in the public domain.  It may  be  used  for
any purpose on any machine by anyone." I would greatly prefer it if *my*
material received no military use.