Bug#17667: An attacker has the opportunity to bypass query logging.
This adds a new, local-only printf format specifier to our *printf functions
that allows us to print known-size buffers that must not be interpreted as
NUL-terminated "strings."
It uses this format-specifier to print to the log, thus fixing this
problem.
fragment is not well-formed xml
Problem:
- ExtractValue silently returned NULL if a wrong XML value is passed.
- In some cases "unexpected END-OF-INPUT" error was not detected, and
a non-NULL result could be returned for a bad XML value.
Fix:
- Adding warning messages, to make user aware why NULL was returned.
- Missing "unexpected END-OF-INPUT" error is reported now.
It was impossible to create some table names on Windows
(e.g. LPT1, AUX, COM1, etc).
Fixed to pad dangerous names with thee "at" signs
(e.g. LPT1@@@, AUX@@@, COM1@@@, and so on).
ExtractValue didn't understand tag and attribute names
consisting of "tricky" national letters (e.g. latin accenter letters).
It happened because XPath lex parser recognized only basic
latin letter a..z ad a part of an identifier.
Fixed to recognize all letters by means of new "full ctype" which
was added recently.
Backporting a 5.0 change:
MAX_BUF was too small for Index.xml
Changeing MAX_BUF and adding assert to easier
catch the same problem in the future.
ctype-extra.c:
Regenerating ctype-extra.c with the fixed conf_to_src.
Buffer was too small to load Index.xml.
So some charsets were not gerenrated.
Making the buffer bigger, and adding an DBUG_ASSERT,
to easier catch the problem in the future.
ctype-extra.c:
Additional charsets were generated.
The table may be corrupted and decimal columns may have invalid values in this case.
To prevent crash we need to check that decimal column has allowable value.
In case of invalid value generate warning and set the value to 0.
A prerequisite for several fulltext and XML bugs.
MY_CHARSET_HANDLER now has a new function "ctype"
to detect a type of the next character in a string
(i.e. digit, letter, space, punctuation, control, etc),
which now works correctly for both 8bit and multibyte charsets.
Previously only 8bit charsets worked correctly,
while any multibyte character was considered as letter
in multibyte charsets.
Many files:
Adding new function
Makefile.am:
Adding build rules for uctypedump,
a dump tool to create my_uctype.h
using Unicode Character Database file.
m_ctype.h:
Adding declaration of my_uni_ctype,
ctype data for Unicode.
Adding new member into MY_CHARSET_HANDLER
Makefile.am:
Adding my_uctype.h into noinst_HEADERS
my_uctype.h, uctypedump.c:
new files:
ctype data for unicode,
and the tool to generate it from
a Unicode Character Database file.
Don't display charsets/collations with HIDDEN flag.
ctype-utf8.c:
Adding HIDDEN flag to "filename"
Changeing ID to 17 which was previosly
used by deprecated "win1251", removed in 4.1.
charset-def.c:
Adding "filename" as a hidden charset, for test purposes.
m_ctype.h:
Adding MY_CS_HIDDEN flag,
to hide charsets and collations from
being displayed in SHOW.
- Encoding itself, implemented as a charset
"filename". Originally planned to use '.'
as an escape character, but now changed to '@'
for two reasons: "ls" does not return
file names starting with '.' considering them
as a kind of hidden files; some platforms
do not allow several dots in a file name.
- replacing many calls of my_snprintf() and
strnxmov() to the new build_table_filename().
- Adding MY_APPEND_EXT mysys flag, to append
an extention rather that replace it.
- Replacing all numeric constants in fn_format
flag arguments to their mysys definitions, e.g.
MY_UNPACK_FILENAME,
- Predictability in several function/methods:
when a table name can appear with or withot .frm
extension. Some functions/methods were changed
so accept names strictly with .frm, other - strictly
without .frm extensions. Several DBUG_ASSERTs were
added to check whether an extension is passed.
Many files:
table name to file name encoding
mysql_priv.h:
Prototypes for new table name encoding tools.
ctype-utf8.c:
Implementing "filename" charset for
table name to file name encoding.
row0mysql.c:
Fixing table name prefix.
mf_format.c:
Adding MY_APPEND_EXT processing.
Many files:
Fixing tests.
my_sys.h:
Adding new flag to append rather than replace an extension.
m_ctype.h:
Adding "filename" charset definition.
When InnoDB compares varchar field in ucs2 with given key using bin collation,
it calls my_strnncollsp_ucs2_bin() to perform comparison.
Because field length was lesser than length of key field should be padded
with trailing spaces in order to get correct result.
Because my_strnncollsp_ucs2_bin() was calling my_strnncollp_ucs2_bin(), which
doesn't pads field, wrong comparison result was returned. This results in
wrong result set.
my_strnncollsp_ucs2_bin() now compares fields like my_strnncollsp_ucs2 do,
but using binary collation.
libmysqld/Makefile.am:
sql/Makefile.am:
Adding new source files.
Adding new file into build process.
include/my_xml.h:
strings/xml.c:
Adding new XML parse flags to skip text normalization and
to use relative tag names. Adding enum for XML token types.
sql/lex.h:
Making parser aware of new SQL functions.
sqll/item_create.h, sql/item_create.cc:
Adding creators for ExtractValue and UpdateXML.
sql/item.h:
Adding new Item types: nodeset and nodeset comparator.
sql/item_xmlfunc.h
sql/item_xmlfunc.cc
Adding new classes implementing XPath functions.
mysql-test/t/xml.test, mysql-test/r/xml.result:
New files: adding test case
into parts when converting to Unicode.
m_ctype.h:
Reorganizing mb_wc return codes to be able
to return "an unassigned N-byte-long character".
sql_string.cc:
Adding code to detect and properly handle
unassigned characters (i.e. the those character
which are correctly formed according to the
character specifications, but don't have Unicode
mapping).
Many files:
Fixing conversion function to return new codes.
ctype_ujis.test, ctype_gbk.test, ctype_big5.test:
Adding a test case.
ctype_ujis.result, ctype_gbk.result, ctype_big5.result:
Fixing results accordingly.
ctype-euc_kr.c:
ctype-gb2312.c:
Adding specific well_formed_length functions
for gb2312 and euckr, to allow storing characters
which are correct according to the character set
specifications but just don't have Unicode mapping.
Previously only those which have Unicode mapping
could be stored, while unassigned characters lead
to data truncation.
Many files:
new file
The table opening process now works the following way:
- Create common TABLE_SHARE object
- Read the .frm file and unpack it into the TABLE_SHARE object
- Create a TABLE object based on the information in the TABLE_SHARE
object and open a handler to the table object
Other noteworthy changes:
- In TABLE_SHARE the most common strings are now LEX_STRING's
- Better error message when table is not found
- Variable table_cache is now renamed 'table_open_cache'
- New variable 'table_definition_cache' that is the number of table defintions that will be cached
- strxnmov() calls are now fixed to avoid overflows
- strxnmov() will now always add one end \0 to result
- engine objects are now created with a TABLE_SHARE object instead of a TABLE object.
- After creating a field object one must call field->init(table) before using it
- For a busy system this change will give you:
- Less memory usage for table object
- Faster opening of tables (if it's has been in use or is in table definition cache)
- Allow you to cache many table definitions objects
- Faster drop of table
Fixing latin1 to cp1252, according to
recent changes in ctype-latin1.c.
Index.xml:
Marking latin1_swedish_ci as "compiled"
to avoid its generating into ctype-extra.c.
ctype-extra.c:
Bug#12076 --with-extra-charsets has no effect
All supported dnamic charsets were generated
from sql/share/charsets/*.xml under #ifdefs.
Compiling is to be activated from "configure"
by adding --with-extra-charsets.
I'm not including auto-recreating of ctype-extra.c
into build process for ease purposes.
field::sort_key() now adds length last for varbinary/blob
VARBINARY/BLOB is now sorted by filesort so that shorter strings comes before longer ones
Fixed issues in test cases from last merge
new file
mysql_fix_privilege_tables.sql, mysql_create_system_tables.sh:
Adding true BINARY/VARBINARY: fixing "password" type, not to be 0x00-padding.
Many files:
Adding true BINARY/VARBINARY: fixing tests not to output 0x00 bytes.
Adding true BINARY/VARBINARY: new pad_char structure member.
ctype-bin.c:
Adding true BINARY/VARBINARY: new pad_char structure member.
New strnxfrm, with two trailing length bytes.
field.cc:
Adding true BINARY/VARBINARY.
Ensure that ccache is also used for C programs
mysql: Ensure that 'delimiter' works the same way in batch mode as in normal mode
mysqldump: Change to use ;; (instead of //) as a stored procedure/trigger delimiter
Fixed test cases by adding missing DROP's and rename views to be of type 'v#'
Removed MY_UNIX_PATH from fn_format()
Removed current_db_used from TABLE_LIST
Removed usage of 'current_thd' in Item_splocal
Removed some compiler warnings
A bit faster longlong2str code
index doesn't return correct result
item_cmpfunc.cc:
Use charset of LIKE to decide whether
to use 8bit or Unicode "escape" value.
But use charset of "escape" to scan escape character.
strings/ctype-xxx.c:
We cannot reduce "end" pointer using charpos(),
because of possible escape characters in the string.
Limit the loop using count of written characters instead.
ctype_like_escape.inc:
new file
mysql-test/t/ctype_xxx:
mysql-test/r/ctype_xxx:
Adding test case.
ctype_latin1.test, ctype_latin1.result:
adding test case
ctype-latin1.c:
Fixing ctype array to treat extended cp1252
letters as valid identifiers on server side,
and as valid "isprint" characters (e.g. on client side).
In cp932, '\' character can be the second byte in a
multi-byte character stream. This makes it difficult to use
mysql_escape_string. Added flag to indicate which languages allow
'\' as second byte of multibyte sequence so that when putting a prepared
statement into the binlog we can decide at runtime whether hex encoding
is really needed.
Changed assembler functions to not access global variables or variables in text segement
Added wrapper function in C to longlong2str() to pass _dig_vec_upper as an argument
ctype-cp932.c:
ctype-gbk.c:
ctype-mb.c:
ctype-simple.c:
ctype-sjis.c:
ctype-ucs2.c:
ctype-ujis.c:
ctype-utf8.c:
Adding explicit cast to return type
in pointer substructions to avoid
warnings from some compilers.
Bug #11987
mysql will truncate the text when the text contain GBK char:"0xA3A0" and "0xA1"
Allow to store and retrieve even unassigned GBK codes.
Like we did in Big5 earlier.
have_gbk.inc, have_gbk.require, ctype_gbk.result, ctype_gbk.test:
new file
Index.xml:
Fixing latin1 comment:
it is actually cp1252, not iso-8859-1
ctype_latin1.result:
changeing test results accordingly.
ctype-latin1.c:
Fixed to- and from-Unicode conversion maps
for better Unicode round trip of undefined
characters.
New BitKeeper file ``mysql-test/include/ctype_innodb_like.inc''
Many files:
bug#11650: LIKE pattern matching using prefix index doesn't return correct result
min and max values were too long in the case of prefix key.
Fix my_like_range functions not to exceed prefix length.
ctype_innodb_like.inc:
new file
Fixing tests accordingly.
ctype-ucs2.c:
The same fix for UCS2.
ctype-utf8.c:
Bug #9557
MyISAM utf8 table crash
The problem was that my_strnncollsp_xxx could
return big value in the range 0..0xffff.
for some constant pairs it could return 32738,
which is defined as MI_FOUND_WRONG_KEY in
myisamdef.h. As a result, table considered to
be crashed.
Fix to return -1,0 or 1.
which will by default exit(1) if malloc() fails, but can be set
to do something else by the calling program does, which mysqld
does to use my_malloc(..., MYF(MY_FAE)) instead. Also checks
allocation in conf_to_src utility program. (Bug #7003)
UPPER/LOWER now can return a string with different length.
mi_test1.c:
Adding new arguments.
Many files:
Changeing caseup/casedn to return a result with different
length than argument.
sql_string.h:
Removing unused method,
mysql_priv.h:
Removing unused method
Gluh's SESSION/GLOBAL for @variables fix in sql_yacc.yy and
Bar's well_formed_len() changes in ndb code
did not make it and should be re-applied manually
fixing test results accordingly.
ctype-uca.c:
It appeared that in traditional Spanish collation
'RR' is not equal to 'R', as Unicode and Mimer state.
We'll go Oracle and IBM way instead:
No special rules to 'RR'.
Windows to call CreateFileMapping() with correct arguments, and
propogating the introduction of query_id_t to everywhere query ids are
passed around. (Bug #8826)
Change string->float conversion to delay division as long as possible.
This gives us more exact integer->float conversion for numbers of type '123.45E+02' (Bug #7740)
Lots of small fixes to multi-precision-math path
Give Note for '123.4e'
Added helper functions type 'val_string_from_real()
Don't give warnings for end space for string2decimal()
Changed storage of values for SP so that we can detect length of argument without strlen()
Changed interface for str2dec() so that we must supple the pointer to the last character in the buffer
fixed end of string detection in string->decimal conversion to avoid false alarm about some string part left unconverted (string can be not null terminated)
ignore my_decimal.cc in libmysqld directory
Renamed HA_VAR_LENGTH to HA_VAR_LENGTH_PART
Renamed in all files FIELD_TYPE_STRING and FIELD_TYPE_VAR_STRING to MYSQL_TYPE_STRING and MYSQL_TYPE_VAR_STRING to make it easy to catch all possible errors
Added support for VARCHAR KEYS to heap
Removed support for ISAM
Now only long VARCHAR columns are changed to TEXT on demand (not CHAR)
Internal temporary files can now use fixed length tables if the used VARCHAR columns are short
- A new flag MY_CS_CSSORT was introduced for case sensitivity.
- Item_func_regexp doesn't substiture ICASE not only
for binary collations but for case sensitive collations as well.
