This commit implements aggregate stored functions. The basic idea behind
the feature is:
* Implement a special instruction FETCH GROUP NEXT ROW that will pause
the execution of the stored function. When the instruction is reached,
execution of the initial query resumes "as if" the function returned.
This gives the server the opportunity to advance to the next row in the
result set.
* Stored aggregates behave like regular aggregate functions. The
implementation of thus resides in the class Item_sum_sp. Because it is
an aggregate function, for each new row in the group, the
Item_sum_sp::add() method will be called. This is when execution resumes
and the function does another iteration to "add" one extra element to
the final result.
* When the end of group is reached, val_xxx() method will be called for
the item. This case is handled by another execute step for the stored
function, only with a special flag to force a call to the return
handler. See Item_sum_sp::execute() for details.
To allow this pause and resume semantic, we must preserve the function
context across executions. This is stored in Item_sp::sp_query_arena only for
aggregate stored functions, but has no impact for regular functions.
We also enforce aggregate functions to include the "FETCH GROUP NEXT ROW"
instruction.
Signed-off-by: Vicențiu Ciorbaru <vicentiu@mariadb.org>
In preparation for implementing custom aggregate functions, refactor
the common code between regular stored functions and aggregate stored
functions. This includes:
* initialising SP result field
* executing a SP
* access checks
In addition, refactor sp_head::execute_function to take two extra
parameters, a function rcontext and a Query_arena. These two paremeters
were initially initialised and destroyed within
sp_head::execute_function, but for aggregate stored functions we will
require control over their lifetime. The owner of these objects now
becomes Item_sp.
Signed-off-by: Vicențiu Ciorbaru <vicentiu@mariadb.org>
Spider patches 026 (MDEV-7723), 031 (MDEV-7727) and 058 (MDEV-12532)
This allows the storage engine to internally compute sum and count
operations.
- Enhance sum items to be able to store the sum value directly.
- return_record_by_parent() is enabled in spider as
HANDLER_HAS_DIRECT_AGGREGATE is defined
- Added spd_environ.h to spider. This is loaded first to ensure that all
MariaDB specific defines that are used by include files are properly
defined.
- This code is tested by the existing spider tests direct_aggregate.test
and direct_aggregate_part.test and also partition.test
1. Removing data type specific constants from enum_item_param_state,
adding SHORT_DATA_VALUE instead.
2. Replacing tests for Item_param::state for the removed constants to
tests for Type_handler::cmp_type() against {INT|REAL|TIME|DECIAML}_RESULT.
Deriving Item_param::PValue from Type_handler_hybrid_field_type,
to store the data type handler of the current value of the parameter.
3. Moving Item_param::decimal_value and Item_param::str_value_ptr
to Item_param::PValue. Adding Item_param::PValue::m_string
and changing Item_param to use it to store string values,
instead of Item::str_value. The intent is to replace Item_param::value
to a st_value based implementation in the future, to avoid duplicate code.
Adding a sub-class Item::PValue_simple, to implement
Item_param::PValue::swap() easier.
Remaming Item_basic_value::fix_charset_and_length_from_str_value()
to fix_charset_and_length() and adding the "CHARSET_INFO" pointer
parameter, instead of getting it directly from item->str_value.charset().
Changing Item_param to pass value.m_string.charset() instead
of str_value.charset().
Adding a String argument to the overloaded
fix_charset_and_length_from_str_value() and changing Item_param
to pass value.m_string instead of str_value.
4. Replacing the case in Item_param::save_in_field() to a call
for Type_handler::Item_save_in_field().
5. Adding new methods into Item_param::PValue:
val_real(), val_int(), val_decimal(), val_str().
Changing the corresponding Item_param methods
to use these new Item_param::PValue methods
internally. Adding a helper method
Item_param::can_return_value() and removing
duplicate code in Item_param::val_xxx().
6. Removing value.set_handler() from Item_param::set_conversion()
and Type_handler_xxx::Item_param_set_from_value().
It's now done inside Item_param::set_param_func(),
Item_param::set_value() and Item_param::set_limit_clause_param().
7. Changing Type_handler_int_result::Item_param_set_from_value()
to set max_length using attr->max_length instead of
MY_INT64_NUM_DECIMAL_DIGITS, to preserve the data type
of the assigned expression more precisely.
8. Adding Type_handler_hybrid_field_type::swap(),
using it in Item_param::PValue::swap().
9. Moving the data-type specific code from
Item_param::query_val_str(), Item_param::eq(),
Item_param::clone_item() to
Item_param::value_query_type_str(),
Item_param::value_eq(), Item_param::value_clone_item(),
to split the "state" dependent code and
the data type dependent code.
Later we'll split the data type related code further
and add new methods in Type_handler. This will be done
after we replace Item_param::PValue to st_value.
10. Adding asserts into set_int(), set_double(), set_decimal(),
set_time(), set_str(), set_longdata() to make sure that
the value set to Item_param corresponds to the previously
set data type handler.
11. Adding tests into t/ps.test and suite/binlog/t/binlog_stm_ps.test,
to cover Item_param::print() and Item_param::append_for_log()
for LIMIT clause parameters.
Note, the patch does not change the behavior covered by the new
tests. Adding for better code coverage.
12. Adding tests for more precise integer data type in queries like this:
EXECUTE IMMEDIATE
'CREATE OR REPLACE TABLE t1 AS SELECT 999999999 AS a,? AS b'
USING 999999999;
The explicit integer literal and the same integer literal
passed as a PS parameter now produce columns of the same data type.
Re-recording old results in ps.result, gis.result, func_hybrid_type.result
accordingly.
- sql_prepare.cc: Moving functions set_param_xxx() as
methods to Item_param
- Replacing a pointer to a function Item_param::set_param_func
to Type_handler based implementation:
Item_param::value now derives from Type_handler_hybrid_field_type.
Adding new virtual methods Type_handler::Item_param_setup_conversion()
and Type_handler::Item_param_set_param_func()
- Moving declaration of some Item_param members from "public:" to "private:"
(CONVERSION_INFO, value, decimal_value)
- Adding a new method Item_param::set_limit_clause_param(),
to share duplicate code, as well as to encapsulate
Item_param::value.
- Adding Item_param::setup_conversion_string() and
Item_param::setup_conversion_blob() to share
the code for binding from a client value
(mysql_stmt_bind_param), and for binding from
an expression (Item).
- Removing two different functions set_param_str_or_null()
and set_param_str(). Adding a common method Item_param::set_param_str().
Item_param::m_empty_string_is_null, used by Item_param::set_param_str().
- Removing the call for setup_one_conversion_function() from
insert_params_from_actual_params_with_log(). It's not needed,
because the call for ps_param->save_in_param() makes sure
to initialized all data type dependent members properly,
by calling setup_conversion_string() from
Type_handler_string_result::Item_param_set_from_value()
and by calling setup_conversion_blob() from
Type_handler_blob_common::Item_param_set_from_value()
- Cleanup: removing multiplication to MY_CHARSET_BIN_MB_MAXLEN
in a few places. It's 1 anyway, and will never change.
TODO:
- Make get_thd_memroot() inline
- To do this, we need to reduce dependence of include files, especially
so that sql_class.h is not depending in item.h
Item_param::set_value() did not set Item::collation and
Item_param::str_value_ptr.str_charset properly. So both
metadata and data for OUT parameters were sent in a wrong
way to the client.
This patch removes the old implementation of Item_param::set_value()
and rewrites it using Type_handler::Item_param_set_from_value(),
so now setting IN and OUT parameters share the a lot of code.
1. Item_param::set_str() now:
- accepts two additional parameters fromcs, tocs
- sets str_value_ptr, to make sure it's always in sync with str_value,
even without Item_param::convert_str_value()
- does collation.set(tocs, DERIVATION_COERCIBLE),
to make sure that DTCollation is valid even without
Item_param::convert_str_value()
2. Item_param::set_value(), which is used to set OUT parameters,
now reuses Type_handler::Item_param_set_from_value().
3. Cleanup: moving Item_param::str_value_ptr to private,
as it's not needed outside.
4. Cleanup: adding a new virtual method
Settable_routine_parameter::get_item_param()
and using it a few new DBUG_ASSERTs, where
Item_param cannot appear.
After this change:
1. Assigning of IN parameters works as before:
a. Item_param::set_str() is called and sets the value as a binary string
b. The original value is sent to the query used for binary/general logging
c. Item_param::convert_str_value() converts the value from the client
character set to the connection character set
2. Assigning of OUT parameters works in the new way:
a. Item_param::set_str() and sets the value
using the source Item's collation, so both Item::collation
and Item_param::str_value_ptr.str_charset are properly set.
b. Protocol_binary::send_out_parameters() sends the
value to the client correctly:
- Protocol::send_result_set_metadata() uses Item::collation.collation
(which is now properly set), to detect if conversion is needed,
and sends a correct collation ID.
- Protocol::send_result_set_row() calls Type_handler::Item_send_str(),
which uses Item_param::str_value_ptr.str_charset
(which is now properly set) to actually perform the conversion.
Make differentiation between pullout for merge and pulout of outer field during exists2in transformation.
In last case the field was outer and so we can safely start from name resolution context of the SELECT where it was pulled.
Old behavior lead to inconsistence between list of tables and outer name resolution context (which skips one SELECT for merge purposes) which creates problem vor name resolution.
As a result of this merge the code for the following tasks appears in 10.3:
- MDEV-12172 Implement tables specified by table value constructors
- MDEV-12176 Transform [NOT] IN predicate with long list of values INTO
[NOT] IN subquery.
Side effect: the second debug Note in cache_temporal_4265.result disappeared.
Before this change:
- During JOIN::cache_const_exprs(),
Item::get_cache() for Item_date_add_interval() was called.
The data type for date_add('2001-01-01',interval 5 day) is VARCHAR,
because the first argument is VARCHAR (not temporal).
Item_get_cache() created Item_cache_str('2001-01-06').
- During evaluate_join_record(), get_datetime_value() was called,
which called Item::get_date() for Item_cache_str('2001-01-06').
This gave the second Note. Then, get_datetime_value() created
a new cache, now Item_cache_temporal for '2001-01-06', so not
further str_to_datetime() happened.
After this change:
- During tem_bool_rowready_func2::fix_length_and_dec(),
Arg_comparator::set_cmp_func_datetime() is called,
which immediately creates an instance of Item_cache_date for
the result of date_add('2001-01-01',interval 5 day).
So later no str_to_datetime happens any more,
neither during JOIN::cache_const_exprs(),
nor during evaluate_join_record().
- Renaming sp_rcontext::sp to sp_rcontext:m_sp for consistency
with other sp_rcontext_members, and for consistency with the
same purpose member Item_sp_variable::m_sp.
- Passing a "const sp_head*" pointer to sp_rcontext::sp_rcontext()
and to sp_rcontext::create().
Initializing sp_rcontext::m_sp right in the constructor
instead of having a separate initialization after "new sp_rcontext"
or sp_rcontext::create().
- Adding the "const" qualifier to sp_rcontext::m_sp and Item_sp_variable::m_sp
Some innobase/xtrabackup changes around from 10.1 are null merged
, in partucular using os_set_file_size to extend tablespaces in server
or mariabackup.
They require non-trivial amount of additional work in 10.2, due to
innobase differences between 10.1 and 10.2
Fixing the data type for the "fuzzydate" parameter to
Item_func_hybrid_field_type::date_op() from uint to ulonglong,
for consistency with Item::get_date().
Changing datatypes for:
- Item_spvar_args::m_table
- sp_rcontext::m_var_table
- return value of create_virtual_tmp_table()
from TABLE* to Virtual_tmp_table*
Advantages:
- Stricter data type control
- Removing the duplicate code (a loop with free_blobs)
from destructors ~sp_rcontext() and ~Item_spvar_args(),
using "delete m_(var_)table" in both instead.
- Using Virtual_tmp_table::delete makes the code call Field::delete,
which calls TRASH() for the freed fields,
which is good for valgrind test runs.
- Implementing stricter data type control for Item_long_func descendants
- Cleanup: renaming Type_handler::can_return_str_ascii() to can_return_text()
(a better name).
- Fix win64 pointer truncation warnings
(usually coming from misusing 0x%lx and long cast in DBUG)
- Also fix printf-format warnings
Make the above mentioned warnings fatal.
- fix pthread_join on Windows to set return value.
