there was two problems about charsets in embedded server
1. mysys/charset.c - defined there default_charset_info variable is
modified by both server and client code (particularly when
--default-charset option is handled)
In embedded server we get two codelines modifying one variable.
I created separate default_client_charset_info for client code
2. mysql->charset and mysql->options.charset initialization isn't
properly done for embedded server - necessary calls added
tables
Currently in INSERT ... SELECT ... LIMIT ... the compiler uses a
temporary table to store the results of SELECT ... LIMIT .. and then
uses that table as a source for INSERT. The problem is that in some cases
it actually skips the LIMIT clause in doing that and materializes the
whole SELECT result set regardless of the LIMIT.
This fix is limiting the process of filling up the temp table with only
that much rows that will be actually used by propagating the LIMIT value.
Certain updates of table joined to self results in unexpected
behavior.
The problem was that record cache was mistakenly enabled for
self-joined table updates. Normally record cache must be disabled
for such updates.
Fixed wrong condition in code that determines whether to use
record cache for self-joined table updates.
Only MyISAM tables were affected.
Fix for bug#16716 for --ps-protocol mode.
item_cmpfunc.cc:
Fix for a memory allocation/freeing problem in agg_cmp_type() after fix
for bug#16377. Few language corrections.
Problem: cast to unsigned limited result to
max signed bigint 9223372036854775808,
instead of max unsigned bigint 18446744073709551615.
Fix: don't use args[0]->val_int() when casting from
a floating point number, use val() instead, with range checkings,
special to unsigned data type.
item_func.cc:
Special handling of cast from REAL_RESULT
to unsigned int: we cannot execute args[0]->val_int()
because it cuts max allowed value to LONGLONG_INT,
instead of ULONGLONG_INT required.
count_distinct3.test:
Getting rid of "Data truncated; out of range ..." warnings.
cast.test, cast.result:
Adding test case.
ps.result:
Fixing that cast from 6570515219.6535
to unsigned didn't round to 6570515220,
and returned 6570515219 instead.
can lead to a wrong result.
All date/time functions has the STRING result type thus their results are
compared as strings. The string date representation allows a user to skip
some of leading zeros. This can lead to wrong comparison result if a date/time
function result is compared to such a string constant.
The idea behind this bug fix is to compare results of date/time functions
and data/time constants as ints, because that date/time representation is
more exact. To achieve this the agg_cmp_type() is changed to take in the
account that a date/time field or an date/time item should be compared
as ints.
This bug fix is partially back ported from 5.0.
The agg_cmp_type() function now accepts THD as one of parameters.
In addition, it now checks if a date/time field/function is present in the
list. If so, it tries to coerce all constants to INT to make date/time
comparison return correct result. The field for the constant coercion is
taken from the Item_field or constructed from the Item_func. In latter case
the constructed field will be freed after conversion of all constant items.
Otherwise the result is same as before - aggregated with help of the
item_cmp_type() function.
From the Item_func_between::fix_length_and_dec() function removed the part
which was converting date/time constants to int if possible. Now this is
done by the agg_cmp_type() function.
The new function result_as_longlong() is added to the Item class.
It indicates that the item is a date/time item and result of it can be
compared as int. Such items are date/time fields/functions.
Correct val_int() methods are implemented for classes Item_date_typecast,
Item_func_makedate, Item_time_typecast, Item_datetime_typecast. All these
classes are derived from Item_str_func and Item_str_func::val_int() converts
its string value to int without regard to the date/time type of these items.
Arg_comparator::set_compare_func() and Arg_comparator::set_cmp_func()
functions are changed to substitute result type of an item with the INT_RESULT
if the item is a date/time item and another item is a constant. This is done
to get a correct result of comparisons like date_time_function() = string_constant.
The bug report revealed two problems related to min/max optimization:
1. If the length of a constant key used in a SARGable condition for
for the MIN/MAX fields is greater than the length of the field an
unwanted warning on key truncation is issued;
2. If MIN/MAX optimization is applied to a partial index, like INDEX(b(4))
than can lead to returning a wrong result set.
3.23 regression test failure
The member SEL_ARG::min_flag was not initialized,
due to which the condition for no GEOM_FLAG in function
key_or did not choose "Range checked for each record" as
the correct access method.
There actually was 3 different problems -
hash_user_connections wasn't cleaned
one strdupped database name wasn't freed
and stmt->mem_root wasn't cleaned as it was
replased with mysql->field_alloc for result
For the last one - i made the library using stmt's
fields to store result if it's the case.
The IN() function uses agg_cmp_type() to aggregate all types of its arguments
to find out some common type for comparisons. In this particular case the
char() and the int was aggregated to double because char() can contain values
like '1.5'. But all strings which do not start from a digit are converted to
0. thus 'a' and 'z' become equal.
This behaviour is reasonable when all function arguments are constants. But
when there is a field or an expression this can lead to false comparisons. In
this case it makes more sense to coerce constants to the type of the field
argument.
The agg_cmp_type() function now aggregates types of constant and non-constant
items separately. If some non-constant items will be found then their
aggregated type will be returned. Thus after the aggregation constants will be
coerced to the aggregated type.
In multi-table delete a table for delete can't be used for selecting in
subselects. Appropriate error was raised but wasn't checked which leads to a
crash at the execution phase.
The mysql_execute_command() now checks for errors before executing select
for multi-delete.
argument can lead to a wrong result.
md5() and sha() functions treat their arguments as case sensitive strings.
But when they are compared their arguments were compared as a case
insensitive strings which leads to two functions with different arguments
and thus different results to being identical. This can lead to a wrong
decision made in the range optimizer and thus lead to a wrong result set.
Item_func_md5::fix_length_and_dec() and Item_func_sha::fix_length_and_dec()
functions now set binary collation on their arguments.
The Item_func_concat::val_str() function tries to make as less re-allocations
as possible. This results in appending strings returned by 2nd and next
arguments to the string returned by 1st argument if the buffer for the first
argument has enough free space. A constant subselect is evaluated only once
and its result is stored in an Item_cache_str. In the case when the first
argument of the concat() function is such a subselect Item_cache_str returns
the stored value and Item_func_concat::val_str() append values of other
arguments to it. But for the next row the value in the Item_cache_str isn't
restored because the subselect is a constant one and it isn't evaluated second
time. This results in appending string values of 2nd and next arguments to the
result of the previous Item_func_concat::val_str() call.
The Item_func_concat::val_str() function now checks whether the first argument
is a constant one and if so it doesn't append values of 2nd and next arguments
to the string value returned by it.
- A segfault occured when the function 'kill_server' called
'my_sigset' with signal number 0. 'my_sigset' is a macro which
uses 'sigaction' to install the signal handler with an invalid
signal number will on most platforms return EINVAL but yields
a segfauilt on IRIX 6.5
- The server crash was detected by mysqld_safe and it was restarted although
a shutdown was requested.
- Semantics of kill_server(0) is not known, leaving it intact
A pattern to generate binlog for DROPped temp table in close_temporary_tables
was buggy: could not deal with a grave-accent-in-name table.
The fix exploits `append_identifier()' for quoting and duplicating accents.
Binlog lacks encoding info about DROPped temporary table.
Idea of the fix is to switch temporary to system_charset_info when a temporary table
is DROPped for binlog. Since that is the server, that automatically, but not the client, who generates the query
the binlog should be updated on the server's encoding for the coming DROP.
The `write_binlog_with_system_charset()' is introduced to replace similar problematic places in the code.
A query with a group by and having clauses could return a wrong
result set if the having condition contained a constant conjunct
evaluated to FALSE.
It happened because the pushdown condition for table with
grouping columns lost its constant conjuncts.
Pushdown conditions are always built by the function make_cond_for_table
that ignores constant conjuncts. This is apparently not correct when
constant false conjuncts are present.
The bug was as follows: When merge_key_fields() encounters "t.key=X OR t.key=Y" it will
try to join them into ref_or_null access via "t.key=X OR NULL". In order to make this
inference it checks if Y<=>NULL, ignoring the fact that value of Y may be not yet known.
The fix is that the check if Y<=>NULL is made only if value of Y is known (i.e. it is a
constant).
TODO: When merging to 5.0, replace used_tables() with const_item() everywhere in merge_key_fields().
The reason of the bug is in that `get_var_with_binlog' performs missed
assingment of
the variables as side-effect. Doing that it eventually calls
`free_underlaid_joins' to pass as an argument `thd->lex->select_lex' of the lex
which belongs to the user query, not
to one which is emulated i.e SET @var1:=NULL.
`get_var_with_binlog' is refined to supply a temporary lex to sql_set_variables's stack.
Problem:
if a user was granted privileges on database "d1",
it also was able to act on "D1" (i.e. in upper case),
even on Unix with case sensitive file system.
Fix:
Initialize grant hash to use binary comparison
if lower_case_file_system is not set (on most unixes),
and case insensitive comparison otherwise (Windows, MacOSX).
mysqldump / SHOW CREATE TABLE will show the NEXT available value for
the PK, rather than the *first* one that was available (that named in
the original CREATE TABLE ... AUTO_INCREMENT = ... statement).
This should produce correct and robust behaviour for the obvious use
cases -- when no data were inserted, then we'll produce a statement
featuring the same value the original CREATE TABLE had; if we dump
with values, INSERTing the values on the target machine should set the
correct next_ID anyway (and if not, we'll still have our AUTO_INCREMENT =
... to do that). Lastly, just the CREATE statement (with no data) for
a table that saw inserts would still result in a table that new values
could safely be inserted to).
There seems to be no robust way however to see whether the next_ID
field is > 1 because it was set to something else with CREATE TABLE
... AUTO_INCREMENT = ..., or because there is an AUTO_INCREMENT column
in the table (but no initial value was set with AUTO_INCREMENT = ...)
and then one or more rows were INSERTed, counting up next_ID. This
means that in both cases, we'll generate an AUTO_INCREMENT =
... clause in SHOW CREATE TABLE / mysqldump. As we also show info on,
say, charsets even if the user did not explicitly give that info in
their own CREATE TABLE, this shouldn't be an issue.
As per above, the next_ID will be affected by any INSERTs that have
taken place, though. This /should/ result in correct and robust
behaviour, but it may look non-intuitive to some users if they CREATE
TABLE ... AUTO_INCREMENT = 1000 and later (after some INSERTs) have
SHOW CREATE TABLE give them a different value (say, CREATE TABLE
... AUTO_INCREMENT = 1006), so the docs should possibly feature a
caveat to that effect.
It's not very intuitive the way it works now (with the fix), but it's
*correct*. We're not storing the original value anyway, if we wanted
that, we'd have to change on-disk representation?
If we do dump/load cycles with empty DBs, nothing will change. This
changeset includes an additional test case that proves that tables
with rows will create the same next_ID for AUTO_INCREMENT = ... across
dump/restore cycles.
Confirmed by support as likely solution for client's problem.
In the code that converts IN predicates to EXISTS predicates it is changing
the select list elements to constant 1. Example :
SELECT ... FROM ... WHERE a IN (SELECT c FROM ...)
is transformed to :
SELECT ... FROM ... WHERE EXISTS (SELECT 1 FROM ... HAVING a = c)
However there can be no FROM clause in the IN subquery and it may not be
a simple select : SELECT ... FROM ... WHERE a IN (SELECT f(..) AS
c UNION SELECT ...) This query is transformed to : SELECT ... FROM ...
WHERE EXISTS (SELECT 1 FROM (SELECT f(..) AS c UNION SELECT ...)
x HAVING a = c) In the above query c in the HAVING clause is made to be
an Item_null_helper (a subclass of Item_ref) pointing to the real
Item_field (which is not referenced anywhere else in the query anymore).
This is done because Item_ref_null_helper collects information whether
there are NULL values in the result. This is OK for directly executed
statements, because the Item_field pointed by the Item_null_helper is
already fixed when the transformation is done. But when executed as
a prepared statement all the Item instances are "un-fixed" before the
recompilation of the prepared statement. So when the Item_null_helper
gets fixed it discovers that the Item_field it points to is not fixed
and issues an error. The remedy is to keep the original select list
references when there are no tables in the FROM clause. So the above
becomes : SELECT ... FROM ... WHERE EXISTS (SELECT c FROM (SELECT f(..)
AS c UNION SELECT ...) x HAVING a = c) In this way c is referenced
directly in the select list as well as by reference in the HAVING
clause. So it gets correctly fixed even with prepared statements. And
since the Item_null_helper subclass of Item_ref_null_helper is not used
anywhere else it's taken out.
Concurrent read and update of privilege structures (like simultaneous
run of SHOW GRANTS and ADD USER) could result in server crash.
Ensure that proper locking of ACL structures is done.
No test case is provided because this bug can't be reproduced
deterministically.
Backporting a changeset made for 5.0. Comments from there:
The fix refines the algorithm of generating DROPs for binlog.
Temp tables with common pseudo_thread_id are clustered into one query.
Consequently one replication event per pseudo_thread_id is generated.
The bug caused wrong result sets for union constructs of the form
(SELECT ... ORDER BY order_list1 [LIMIT n]) ORDER BY order_list2.
For such queries order lists were concatenated and limit clause was
completely neglected.
Fixing part2 of this problem: AND didn't work well
with utf8_czech_ci and utf8_lithianian_ci in some cases.
The problem was because when during condition optimization
field was replaced with a constant, the constant's collation
and collation derivation was used later for comparison instead
of the field collation and derivation, which led to non-equal
new condition in some cases.
This patch copies collation and derivation from the field being removed
to the new constant, which makes comparison work using the same collation
with the one which would be used if no condition optimization were done.
In other words:
where s1 < 'K' and s1 = 'Y';
was rewritten to:
where 'Y' < 'K' and s1 = 'Y';
Now it's rewritten to:
where 'Y' collate collation_of_s1 < 'K' and s1 = 'Y'
(using derivation of s1)
Note, the first problem of this bug (with latin1_german2_ci) was fixed
earlier in 5.0 tree, in a separate changeset.
After a locking error the open table(s) were not fully
cleaned up for reuse. But they were put into the open table
cache even before the lock was tried. The next statement
reused the table(s) with a wrong lock type set up. This
tricked MyISAM into believing that it don't need to update
the table statistics. Hence CHECK TABLE reported a mismatch
of record count and table size.
Fortunately nothing worse has been detected yet. The effect
of the test case was that the insert worked on a read locked
table. (!)
I added a new function that clears the lock type from all
tables that were prepared for a lock. I call this function
when a lock failes.
No test case. One test would add 50 seconds to the
test suite. Another test requires file mode modifications.
I added a test script to the bug report. It contains three
cases for failing locks. All could reproduce a table
corruption. All are fixed by this patch.
This bug was not lock timeout specific.
Conversion from int and real numbers to UCS2 didn't work fine:
CONVERT(100, CHAR(50) UNICODE)
CONVERT(103.9, CHAR(50) UNICODE)
The problem appeared because numbers have binary charset, so,
simple charset recast binary->ucs2 was performed
instead of real conversion.
Fixed to make numbers pretend to be non-binary.
used
In a simple queries a result of the GROUP_CONCAT() function was always of
varchar type.
But if length of GROUP_CONCAT() result is greater than 512 chars and temporary
table is used during select then the result is converted to blob, due to
policy to not to store fields longer than 512 chars in tmp table as varchar
fields.
In order to provide consistent behaviour, result of GROUP_CONCAT() now
will always be converted to blob if it is longer than 512 chars.
Item_func_group_concat::field_type() is modified accordingly.
too many open statements". The patch adds a new global variable
@@max_prepared_stmt_count. This variable limits the total number
of prepared statements in the server. The default value of
@@max_prepared_stmt_count is 16382. 16382 small statements
(a select against 3 tables with GROUP, ORDER and LIMIT) consume
100MB of RAM. Once this limit has been reached, the server will
refuse to prepare a new statement and return ER_UNKNOWN_ERROR
(unfortunately, we can't add new errors to 4.1 without breaking 5.0). The limit is changeable after startup
and can accept any value from 0 to 1 million. In case
the new value of the limit is less than the current
statement count, no new statements can be added, while the old
still can be used. Additionally, the current count of prepared
statements is now available through a global read-only variable
@@prepared_stmt_count.
gives wrong results". Implement previously missing
Item_row::cleanup. The bug is not repeatable in 5.0, probably
due to a coincidence: the problem is present in 5.0 as well.
The GROUP_CONCAT uses its own temporary table. When ROLLUP is present
it creates the second copy of Item_func_group_concat. This copy receives the
same list of arguments that original group_concat does. When the copy is
set up the result_fields of functions from the argument list are reset to the
temporary table of this copy.
As a result of this action data from functions flow directly to the ROLLUP copy
and the original group_concat functions shows wrong result.
Since queries with COUNT(DISTINCT ...) use temporary tables to store
the results the COUNT function they are also affected by this bug.
The idea of the fix is to copy content of the result_field for the function
under GROUP_CONCAT/COUNT from the first temporary table to the second one,
rather than setting result_field to point to the second temporary table.
To achieve this goal force_copy_fields flag is added to Item_func_group_concat
and Item_sum_count_distinct classes. This flag is initialized to 0 and set to 1
into the make_unique() member function of both classes.
To the TMP_TABLE_PARAM structure is modified to include the similar flag as
well.
The create_tmp_table() function passes that flag to create_tmp_field().
When the flag is set the create_tmp_field() function will set result_field
as a source field and will not reset that result field to newly created
field for Item_func_result_field and its descendants. Due to this there
will be created copy func to copy data from old result_field to newly
created field.
table.cc:
Fixing to use system_charset_info instead of default_charset_info.
Crash happened because the "ctype" array is empty in UCS2,
and thus cannot be used with my_isspace().
The reason why UCS2 appeared in this context was because of
of default_charset_info variable incorrectly substituted to my_isspace().
As functions check_db_name(), check_table_name() and check_column_name()
always get values in utf8, system_charset_info must be used instead.
ctype_ucs2_def.test, ctype_ucs2_def-master.opt, ctype_ucs2_def.result:
new file
to binlog too much.
When InnoDB has to rollback a transaction because the lock table has
filled up, it also needs to inform the upper layer that the transaction
was rolled back so that the cached transaction is not written to the
binary log.
union.result, union.test:
Adding test case.
item.cc:
Allow safe character set conversion in UNION
- string constant to column's charset
- to unicode
Thus, UNION now works the same with CONCAT (and other string functions)
in respect of aggregating arguments with different character sets.
column is increasing when table is recreated with PS/SP":
make use of create_field::char_length more consistent in the code.
Reinit create_field::length from create_field::char_length
for every execution of a prepared statement (actually fixes the
bug).
Bug #17257 ndb, update fails for inner joins if tables do not have Primary Key
change: the allocated area by setValue may not be around for later, store hidden key in special member variable instead
Bug #17158 load data infile of char values into table of char with no (PK) fails to load
Bug #17081 Doing "LOAD DATA INFILE" directly after delete can cause missing data
- Change "mysql_create_db" to not call "send_ok" if in silent mode i.e. called from "load_master_data"
- Change mysqltest to detect when there aren't as many warnings available as was reported.
When setup_fields() function finds field named '*' it expands it to the list
of all table fields. It does so by checking that the first char of
field_name is '*', but it doesn't checks that the '* is the only char.
Due to this, when updating table with a field named like '*name', such field
is wrongly treated as '*' and expanded. This leads to making list of fields
to update being longer than list of the new values. Later, the fill_record()
function crashes by dereferencing null when there is left fields to update,
but no more values.
Added check in the setup_fields() function which ensures that the field
expanding will be done only when '*' is the only char in the field name.
fix for bug #15828 after review
doing val_str now before testing of null value secures the function for null values returned by dynamic functions - the fix before was incomplete andy covered constant null values
Remove wrong fix for Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
Safety fix for bug #13855 "select distinct with group by caused server crash"
when high concurrency": remove HASH::current_record and make it
an external search parameter, so that it can not be the cause of a
race condition under high concurrent load.
The bug was in a race condition in table_hash_search,
when column_priv_hash.current_record was overwritten simultaneously
by multiple threads, causing the search for a suitable grant record
to fail.
No test case as the bug is repeatable only under concurrent load.
field.cc:
BLOB variations have number-in-bytes limit,
unlike CHAR/VARCHAR which have number-of-characters limits.
A tinyblob column can store up to 255 bytes.
In the case of basic Latin letters (which use 1 byte per character)
we can store up to 255 characters in a tinyblob column.
When passing an utf8 tinyblob column as an argument into
a function (e.g. COALESCE) we need to reserve 3*255 bytes.
I.e. multiply length in bytes to mbcharlen for the character set.
Although in reality a tinyblob column can never be 3*255 bytes long,
we need to set max_length to multiply to make fix_length_and_dec()
of the function-caller (e.g. COALESCE) calculate the correct max_length
for the column being created.
ctype_utf8.result, ctype_utf8.test:
Adding test case.
table' lockup".
Changes from the innodb-4.1-ss11 snapshot.
Do not call os_file-create_tmpfile() at runtime. Instead, create
a tempfile at startup and guard access to it with a mutex.
Also, fix bugs:
10511: "Wrong padding of UCS2 CHAR columns in ON UPDATE CASCADE";
13778: "If FOREIGN_KEY_CHECKS=0, one can create inconsistent FOREIGN
KEYs". When FOREIGN_KEY_CHECKS=0 we still need to check that
datatypes between foreign key references are compatible.
Also, added test cases (also for bug 9802).
into parts when converting to Unicode.
m_ctype.h:
Reorganizing mb_wc return codes to be able
to return "an unassigned N-byte-long character".
sql_string.cc:
Adding code to detect and properly handle
unassigned characters (i.e. the those character
which are correctly formed according to the
character specifications, but don't have Unicode
mapping).
Many files:
Fixing conversion function to return new codes.
ctype_ujis.test, ctype_gbk.test, ctype_big5.test:
Adding a test case.
ctype_ujis.result, ctype_gbk.result, ctype_big5.result:
Fixing results accordingly.
Problem #1: INSERT...SELECT, Version for 4.1.
INSERT ... SELECT with the same table on both sides (hidden
below a MERGE table) does now work by buffering the select result.
The duplicate detection works now after open_and_lock_tables()
on the locks.
I did not find a test case that failed without the change in
sql_update.cc. I made the change anyway as it should in theory
fix a possible MERGE table problem with multi-table update.
latin5.xml:
- Fixing order thee Turkish letters to conform Turkish rules.
- All non-Turkish accented letters are mappend
to their non-accented counterparts.
- Set max_length of Item_func_uuid to max_length*system_charset_info->mbmaxlen
Note! Item_func_uuid should be set to use 'ascii' charset when hex(), format(), md5()
etc will use 'ascii'
- Comitting again, the old patch seems to have been lost.
field.cc:
Adding longlong range checking to return
LONGLONG_MIN or LONGLONG_MAX when out of range.
Using (longlong) cast only when range is ok.
cast.test:
Adding test case.
cast.result:
Fixing results accordingly.
depending on table order
multi_update::send_data() was counting updates, not updated rows. Thus if one
record have several updates it will be counted several times in 'rows matched'
but updated only once.
multi_update::send_data() now counts only unique rows.
Problem #1: INSERT...SELECT
INSERT ... SELECT with the same table on both sides (hidden
below a MERGE table) does now work by buffering the select result.
The duplicate detection works now after open_and_lock_tables()
on the locks.
I did not find a test case that failed without the change in
sql_update.cc. I made the change anyway as it should in theory
fix a possible MERGE table problem with multi-table update.
CREATE TABLE and PS/SP": make sure that 'typelib' object for
ENUM values and 'Item_string' object for DEFAULT clause are
created in the statement memory root.
