query / no aggregate of subquery
The optimizer counts the aggregate functions that
appear as top level expressions (in all_fields) in
the current subquery. Later it makes a list of these
that it uses to actually execute the aggregates in
end_send_group().
That count is used in several places as a flag whether
there are aggregates functions.
While collecting the above info it must not consider
aggregates that are not aggregated in the current
context. It must treat them as normal expressions
instead. Not doing that leads to incorrect data about
the query, e.g. running a query that actually has no
aggregate functions as if it has some (and hence is
expected to return only one row).
Fixed by ignoring the aggregates that are not aggregated
in the current context.
One other smaller omission discovered and fixed in the
process : the place of aggregation was not calculated for
user defined functions. Fixed by calling
Item_sum::init_sum_func_check() and
Item_sum::check_sum_func() as it's done for the rest of
the aggregate functions.
context was used as an argument of GROUP_CONCAT.
Ensured correct setting of the depended_from field in references
generated for set functions aggregated in outer selects.
A wrong value of this field resulted in wrong maps returned by
used_tables() for these references.
Made sure that a temporary table field is added for any set function
aggregated in outer context when creation of a temporary table is
needed to execute the inner subquery.
To correctly decide which predicates can be evaluated with a given table
the optimizer must know the exact set of tables that a predicate depends
on. If that mask is too wide (refer to non-existing tables) the optimizer
can erroneously skip a predicate.
One such case of wrong table usage mask were the aggregate functions.
The have a all-1 mask (meaning depend on all tables, including non-existent
ones).
Fixed by making a real used_tables mask for the aggregates. The mask is
constructed in the following way :
1. OR the table dependency masks of all the arguments of the aggregate.
2. If all the arguments of the function are from the local name resolution
context and it is evaluated in the same name resolution
context where it is referenced all the tables from that name resolution
context are OR-ed to the dependency mask. This is to denote that an
aggregate function depends on the number of rows it processes.
3. Handle correctly the case of an aggregate function optimization (such that
the aggregate function can be pre-calculated and made a constant).
Made sure that an aggregate function is never a constant (unless subject of a
specific optimization and pre-calculation).
One other flaw was revealed and fixed in the process : references were
not calling the recalculation method for used_tables of their targets.
Corrected spelling in copyright text
Makefile.am:
Don't update the files from BitKeeper
Many files:
Removed "MySQL Finland AB & TCX DataKonsult AB" from copyright header
Adjusted year(s) in copyright header
Many files:
Added GPL copyright text
Removed files:
Docs/Support/colspec-fix.pl
Docs/Support/docbook-fixup.pl
Docs/Support/docbook-prefix.pl
Docs/Support/docbook-split
Docs/Support/make-docbook
Docs/Support/make-makefile
Docs/Support/test-make-manual
Docs/Support/test-make-manual-de
Docs/Support/xwf
When only one row was present, the subtraction of nearly the same number
resulted in catastropic cancellation, introducing an error in the
VARIANCE calculation near 1e-15. That was sqrt()ed to get STDDEV, the
error was escallated to near 1e-8.
The simple fix of testing for a row count of 1 and forcing that to yield
0.0 is insufficient, as two rows of the same value should also have a
variance of 0.0, yet the error would be about the same.
So, this patch changes the formula that computes the VARIANCE to be one
that is not subject to catastrophic cancellation.
In addition, it now uses only (faster-than-decimal) floating point numbers
to calculate, and renders that to other types on demand.
When implicitly converting string fields to numbers the
string-to-number conversion error was not sent to the client.
Added code to send the conversion error as warning.
We also need to prevent generation of warnings from the places
where val_xxx() methods are called for the sole purpose of updating
the Item::null_value flag.
To achieve that a special function is added (and called) :
update_null_value(). This function will set the no_errors flag and
will call val_xxx(). The warning generation in Field_string::val_xxx()
will use the flag when generating the conversion warnings.
Item::val_xxx() may be called by the server several times at execute time
for a single query. Calls to val_xxx() may be very expensive and sometimes
(count(distinct), sum(distinct), avg(distinct)) not possible.
To avoid that problem the results of calculation for these aggregate
functions are cached so that val_xxx() methods just return the calculated
value for the second and subsequent calls.
(COUNT(*) = 1) not working in SELECT inside prepared statement.
Note: the warning was introduced in 5.0 and 5.1, 4.1 is OK with the
original fix.
The problem was that in 5.0 and 5.1 clear() for group functions may
access hybrid_type member, and this member is initialized in
fix_fields().
So we should not call clear() from item cleanup() methods, as cleanup()
may be called for unfixed items.
statement.
The problem was that during statement re-execution if the result was
empty the old result could be returned for group functions.
The solution is to implement proper cleanup() method in group
functions.
used
In a simple queries a result of the GROUP_CONCAT() function was always of
varchar type.
But if length of GROUP_CONCAT() result is greater than 512 chars and temporary
table is used during select then the result is converted to blob, due to
policy to not to store fields longer than 512 chars in tmp table as varchar
fields.
In order to provide consistent behaviour, result of GROUP_CONCAT() now
will always be converted to blob if it is longer than 512 chars.
Item_func_group_concat::field_type() is modified accordingly.
The GROUP_CONCAT uses its own temporary table. When ROLLUP is present
it creates the second copy of Item_func_group_concat. This copy receives the
same list of arguments that original group_concat does. When the copy is
set up the result_fields of functions from the argument list are reset to the
temporary table of this copy.
As a result of this action data from functions flow directly to the ROLLUP copy
and the original group_concat functions shows wrong result.
Since queries with COUNT(DISTINCT ...) use temporary tables to store
the results the COUNT function they are also affected by this bug.
The idea of the fix is to copy content of the result_field for the function
under GROUP_CONCAT/COUNT from the first temporary table to the second one,
rather than setting result_field to point to the second temporary table.
To achieve this goal force_copy_fields flag is added to Item_func_group_concat
and Item_sum_count_distinct classes. This flag is initialized to 0 and set to 1
into the make_unique() member function of both classes.
To the TMP_TABLE_PARAM structure is modified to include the similar flag as
well.
The create_tmp_table() function passes that flag to create_tmp_field().
When the flag is set the create_tmp_field() function will set result_field
as a source field and will not reset that result field to newly created
field for Item_func_result_field and its descendants. Due to this there
will be created copy func to copy data from old result_field to newly
created field.
used
In a simple queries a result of the GROUP_CONCAT() function was always of
varchar type.
But if length of GROUP_CONCAT() result is greater than 512 chars and temporary
table is used during select then the result is converted to blob, due to
policy to not to store fields longer than 512 chars in tmp table as varchar
fields.
In order to provide consistent behaviour, result of GROUP_CONCAT() now
will always be converted to blob if it is longer than 512 chars.
Item_func_group_concat::field_type() is modified accordingly.
Added test cases for bug #12863.
item_sum.cc, item_sum.h:
Fixed bug #12863.
Added a flag to Item_func_group_concat set to FALSE after
concatenation of the first element of a group.
(aka "deinit is not called when calling udf from trigger").
We should call udf_deinit() function during cleanup phase after prepared
(or ordinary) statement execution instead of calling it from Item's
desctructor.
No test case is provided since it is hard to test UDF's from our test
suite.
Change string->float conversion to delay division as long as possible.
This gives us more exact integer->float conversion for numbers of type '123.45E+02' (Bug #7740)
Added a test case for bug #7769.
item_sum.h:
Fixed bug #7769: a crash for queries with group_concat and
having when the query table was empty.
The bug was due an unsafe dereferencing.
New mysqltest that can run mysqltest with PS
Added support for ZEROFILL in PS
Fixed crash when one called mysql_stmt_store_result() without a preceding mysql_stmt_bind_result()
Updated test cases to support --ps-protocol
(Some tests are still run using old protocol)
Fixed crash in PS when using SELECT * FROM t1 NATURAL JOIN t2...
Fixed crash in PS when using sub queries
Create table didn't signal when table was created. This could cause a "DROP TABLE created_table" in another thread to wait "forever"
Fixed wrong permissions check in PS and multi-table updates (one could get permission denied for legal quries)
Fix for PS and SELECT ... PROCEDURE
Reset all warnings when executing a new PS query
group_concat(...ORDER BY) didn't work with PS
Fixed problem with test suite when not using innodb
a second time". The bug was caused by incompatibility of
negations elimination algorithm and PS: during first statement
execute a subtree with negation was replaced with equivalent
subtree without NOTs.
The problem was that although this transformation was permanent,
items of the new subtree were created in execute-local memory.
The patch adds means to check if it is the first execute of a
prepared statement, and if this is the case, to allocate items
in memory of the prepared statement.
The implementation:
- backports Item_arena from 5.0
- adds Item_arena::is_stmt_prepare(),
Item_arena::is_first_stmt_execute().
- deletes THD::allocate_temporary_pool_for_ps_preparing(),
THD::free_temporary_pool_for_ps_preparing(); they
were redundant.
and adds a few invariants:
- thd->free_list never contains junk (= freed items)
- thd->current_arena is never null. If there is no
prepared statement, it points at the thd.
The rest of the patch contains mainly mechanical changes and
cleanups.
