mariadb/strings
Mattias Jonsson d92a7cb76a Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING
Due to an internal change in the server code in between 5.1 and 5.5
(wl#2649) the hash function used in KEY partitioning changed
for numeric and date/time columns (from binary hash calculation
to character based hash calculation).

Also enum/set changed from latin1 ci based hash calculation to
binary hash between 5.1 and 5.5. (bug#11759782).

These changes makes KEY [sub]partitioned tables on any of
the affected column types incompatible with 5.5 and above,
since the calculation of partition id differs.

Also since InnoDB asserts that a deleted row was previously
read (positioned), the server asserts on delete of a row that
is in the wrong partition.

The solution for this situation is:

1) The partitioning engine will check that delete/update will go to the
partition the row was read from and give an error otherwise, consisting
of the rows partitioning fields. This will avoid asserts in InnoDB and
also alert the user that there is a misplaced row. A detailed error
message will be given, including an entry to the error log consisting
of both table name, partition and row content (PK if exists, otherwise
all partitioning columns).


2) A new optional syntax for KEY () partitioning in 5.5 is allowed:
[SUB]PARTITION BY KEY [ALGORITHM = N] (list_of_cols)
Where N = 1 uses the same hashing as 5.1 (Numeric/date/time fields uses
binary hashing, ENUM/SET uses charset hashing) N = 2 uses the same
hashing as 5.5 (Numeric/date/time fields uses charset hashing,
ENUM/SET uses binary hashing). If not set on CREATE/ALTER it will
default to 2.

This new syntax should probably be ignored by NDB.


3) Since there is a demand for avoiding scanning through the full
table, during upgrade the ALTER TABLE t PARTITION BY ... command is
considered a no-op (only .frm change) if everything except ALGORITHM
is the same and ALGORITHM was not set before, which allows manually
upgrading such table by something like:
ALTER TABLE t PARTITION BY KEY ALGORITHM = 1 () or
ALTER TABLE t PARTITION BY KEY ALGORITHM = 2 ()


4) Enhanced partitioning with CHECK/REPAIR to also check for/repair
misplaced rows. (Also works for ALTER TABLE t CHECK/REPAIR PARTITION)

CHECK FOR UPGRADE:
If the .frm version is < 5.5.3
and uses KEY [sub]partitioning
and an affected column type
then it will fail with an message:
KEY () partitioning changed, please run:
ALTER TABLE `test`.`t1`  PARTITION BY KEY ALGORITHM = 1 (a)
PARTITIONS 12
(i.e. current partitioning clause, with the addition of
ALGORITHM = 1)

CHECK without FOR UPGRADE:
if MEDIUM (default) or EXTENDED options are given:
Scan all rows and verify that it is in the correct partition.
Fail for the first misplaced row.

REPAIR:
if default or EXTENDED (i.e. not QUICK/USE_FRM):
Scan all rows and every misplaced row is moved into its correct
partitions.


5) Updated mysqlcheck (called by mysql_upgrade) to handle the
new output from CHECK FOR UPGRADE, to run the ALTER statement
instead of running REPAIR.

This will allow mysql_upgrade (or CHECK TABLE t FOR UPGRADE) to upgrade
a KEY [sub]partitioned table that has any affected field type
and a .frm version < 5.5.3 to ALGORITHM = 1 without rebuild.


Also notice that if the .frm has a version of >= 5.5.3 and ALGORITHM
is not set, it is not possible to know if it consists of rows from
5.1 or 5.5! In these cases I suggest that the user does:
(optional)
LOCK TABLE t WRITE;
SHOW CREATE TABLE t;
(verify that it has no ALGORITHM = N, and to be safe, I would suggest
backing up the .frm file, to be used if one need to change to another
ALGORITHM = N, without needing to rebuild/repair)
ALTER TABLE t <old partitioning clause, but with ALGORITHM = N>;
which should set the ALGORITHM to N (if the table has rows from
5.1 I would suggest N = 1, otherwise N = 2)
CHECK TABLE t;
(here one could use the backed up .frm instead and change to a new N
and run CHECK again and see if it passes)
and if there are misplaced rows:
REPAIR TABLE t;
(optional)
UNLOCK TABLES;
2013-01-30 17:51:52 +01:00
..
bchange.c
bmove_upp.c
CHARSET_INFO.txt
CMakeLists.txt
conf_to_src.c
ctype-big5.c
ctype-bin.c BUG#14303860 - EXECUTING A SELECT QUERY WITH TOO 2013-01-14 16:51:52 +05:30
ctype-cp932.c
ctype-czech.c
ctype-euc_kr.c
ctype-eucjpms.c
ctype-extra.c
ctype-gb2312.c
ctype-gbk.c
ctype-latin1.c
ctype-mb.c BUG#14303860 - EXECUTING A SELECT QUERY WITH TOO 2013-01-14 16:51:52 +05:30
ctype-simple.c BUG#14303860 - EXECUTING A SELECT QUERY WITH TOO 2013-01-14 16:51:52 +05:30
ctype-sjis.c
ctype-tis620.c
ctype-uca.c BUG#14303860 - EXECUTING A SELECT QUERY WITH TOO 2013-01-14 16:51:52 +05:30
ctype-ucs2.c
ctype-ujis.c
ctype-utf8.c BUG#14303860 - EXECUTING A SELECT QUERY WITH TOO 2013-01-14 16:51:52 +05:30
ctype-win1250ch.c
ctype.c BUG#14303860 - EXECUTING A SELECT QUERY WITH TOO 2013-01-14 16:51:52 +05:30
decimal.c Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING 2013-01-30 17:51:52 +01:00
do_ctype.c
dtoa.c
dump_map.c
int2str.c
is_prefix.c
latin2.def
llstr.c
longlong2str.c
my_strchr.c
my_strtoll10.c
my_vsnprintf.c
README
str2int.c
str_alloc.c
strappend.c
strcend.c
strcont.c
strend.c
strfill.c
string.doc
strmake.c
strmov.c
strnlen.c
strnmov.c
strxmov.c
strxnmov.c
t_ctype.h
uca-dump.c
uctypedump.c
utr11-dump.c
xml.c

File   : README
Author : Richard A. O'Keefe.
Updated: 30 April 1984
Purpose: Explain the new strings package.

    The UNIX string libraries (described in the string(3) manual page)
differ from UNIX to UNIX (e.g. strtok is not in V7 or 4.1bsd).  Worse,
the sources are not in the public domain, so that if there is a string
routine which is nearly what you want but not quite you can't  take  a
copy  and  modify it.  And of course C programmers on non-UNIX systems
are at the mercy of their supplier.

    This package was designed to let me do reasonable things with  C's
strings  whatever UNIX (V7, PaNiX, UX63, 4.1bsd) I happen to be using.
Everything in the System III manual is here and does just what the  S3
manual  says  it does.  There are also lots of new goodies.  I'm sorry
about the names, but the routines do have to work  on  asphyxiated-at-
birth  systems  which  truncate identifiers.  The convention is that a
routine is called
 str [n] [c] <operation>
If there is an "n", it means that the function takes an (int) "length"
argument, which bounds the number of characters to be moved or  looked
at.  If the function has a "set" argument, a "c" in the name indicates
that  the complement of the set is used.  Functions or variables whose
names start with _ are support routines which aren't really meant  for
general  use.  I don't know what the "p" is doing in "strpbrk", but it
is there in the S3 manual so it's here too.  "istrtok" does not follow
this rule, but with 7 letters what can you do?

    I have included new versions of atoi(3) and atol(3) as well.  They
use a new primitive str2int, which takes a pair of bounds and a radix,
and does much more thorough checking than the normal atoi and atol do.
The result returned by atoi & atol is valid if and only if errno == 0.
There is also an output conversion routine int2str, with itoa and ltoa
as interface macros.  Only after writing int2str did I notice that the
str2int routine has no provision for unsigned numbers.  On reflection,
I don't greatly care.   I'm afraid that int2str may depend on your "C"
compiler in unexpected ways.  Do check the code with -S.

    Several of these routines have "asm" inclusions conditional on the
VaxAsm option.  These insertions can make the routines which have them
quite a bit faster, but there is a snag.  The VAX architects, for some
reason best known to themselves and their therapists, decided that all
"strings" were shorter than 2^16 bytes.  Even when the length operands
are in 32-bit registers, only 16 bits count.  So the "asm" versions do
not work for long strings.  If you can guarantee that all your strings
will be short, define VaxAsm in the makefile, but in general, and when
using other machines, do not define it.

    To use this library, you need the "strings.a" library file and the
"strings.h" and "ctypes.h" header files.  The other header  files  are
for compiling the library itself, though if you are hacking extensions
you  may  find  them useful.  General users really shouldn't see them.
I've defined a few macros I find useful in "strings.h"; if you have no
need for "index", "rindex", "streql", and "beql", just edit them  out.
On the 4.1bsd system I am using declaring all these functions 'extern'
does not mean that they will all be loaded; but only the ones you use.
When using lesser systems you may find it necessary to break strings.h
up, or you could get by with just adding "extern" declarations for the
functions you want as you need them.  Many of these functions have the
same names as functions in the "standard C library", by design as this
is a replacement/reimplementation of part of that library.  So you may
have to talk the loader into loading this library first.   Again, I've
found no problems on 4.1bsd.

    You may wonder at my failure to provide manual pages for this code.
For the things in V7, 4.?, or SIII, you should be able to use whichever
manual page came with that system,  and anything I might write would be
so like it as to raise suspicions of violating AT&T copyrights.  In the
sources you will find comments which provide far more documentation for
these routines than AT&T ever provided for their strings stuff,  I just
don't happen to have put it in nroff -man form.   Had I done so, the .3
files would have outbulked the .c files!

    These files are in the public domain.  This includes getopt.c, which
is the work of Henry Spencer, University of Toronto Zoology, who says of
it "None of this software is derived from Bell software. I had no access
to the source for Bell's versions at the time I wrote it.  This software
is hereby explicitly placed in the public domain.  It may  be  used  for
any purpose on any machine by anyone." I would greatly prefer it if *my*
material received no military use.