Adding support for the ROW data type in the stored function RETURNS clause:
- explicit ROW(..members...) for both sql_mode=DEFAULT and sql_mode=ORACLE
CREATE FUNCTION f1() RETURNS ROW(a INT, b VARCHAR(32)) ...
- anchored "ROW TYPE OF [db1.]table1" declarations for sql_mode=DEFAULT
CREATE FUNCTION f1() RETURNS ROW TYPE OF test.t1 ...
- anchored "[db1.]table1%ROWTYPE" declarations for sql_mode=ORACLE
CREATE FUNCTION f1() RETURN test.t1%ROWTYPE ...
Adding support for anchored scalar data types in RETURNS clause:
- "TYPE OF [db1.]table1.column1" for sql_mode=DEFAULT
CREATE FUNCTION f1() RETURNS TYPE OF test.t1.column1;
- "[db1.]table1.column1" for sql_mode=ORACLE
CREATE FUNCTION f1() RETURN test.t1.column1%TYPE;
Details:
- Adding a new sql_mode_t parameter to
sp_head::create()
sp_head::sp_head()
sp_package::create()
sp_package::sp_package()
to guarantee early initialization of sp_head::m_sql_mode.
Before this change, this member was not initialized at all during
CREATE FUNCTION/PROCEDURE/PACKAGE statements, and was not used.
Now it needs to be initialized to write properly the
mysql.proc.returns column, according to the create time sql_mode.
- Code refactoring to make the things simpler and functions smaller:
* Adding a new method
Field_row::row_create_fields(THD *thd, List<Spvar_definition> *list)
to make a Virtual_tmp_table with Fields for ROW members
from an explicit definition.
* Adding a new method
Field_row::row_create_fields(THD *thd, const Spvar_definition &def)
to make a Virtual_tmp_table with Fields for ROW members
from an explicit or a table anchored definition.
* Adding a new method
Item_args::add_array_of_item_field(THD *thd, const Virtual_tmp_table &vtable)
to create and array of Item_field corresponding to all Field instances
in a Virtual_tmp_table
* Removing Item_field_row::row_create_items(). It was decomposed
into the new methods described above.
* Moving the code from the loop body in sp_rcontext::init_var_items()
into a separate method Spvar_definition::make_item_field_row(),
to make the code clearer (smaller functions).
make_item_field_row() itself uses the new methods described above.
- Changing the data type of sp_head::m_return_field_def
from Column_definition to Spvar_definition.
So now it supports not only SQL column field types,
but also explicit ROW and anchored ROW data types,
as well as anchored column types.
- Adding a new Column_definition parameter to sp_head::create_result_field().
Before this patch, create_result_field() took the definition only
from m_return_field_def. Now it's also called with a local Column_definition
variable which contains the explicit definition resolved from an
anchored defition.
- Modifying sql_yacc.yy to support the new grammar.
Adding new helper methods:
* sf_return_fill_definition_row()
* sf_return_fill_definition_rowtype_of()
* sf_return_fill_definition_type_of()
- Fixing tests in:
* Virtual_tmp_table::setup_field_pointers() in sql_select.cc
* Send_field::normalize() in field.h
* store_column_type()
to prevent calling Type_handler_row::field_type(),
which is implemented a DBUG_ASSERT(0).
Before this patch the affected methods and functions were called only
for scalar data types. Now ROW is also possible.
- Adding a new virtual method Field::cols()
- Overriding methods:
Item_func_sp::cols()
Item_func_sp::element_index()
Item_func_sp::check_cols()
Item_func_sp::bring_value()
to support the ROW data type.
- Extending the rule sp_return_type to support
* explicit ROW and anchored ROW data types
* anchored scalar data types
- Overriding Field_row::sql_type() to print
the data type of an explicit ROW.
This patch also fixes:
MDEV-33050 Build-in schemas like oracle_schema are accent insensitive
MDEV-33084 LASTVAL(t1) and LASTVAL(T1) do not work well with lower-case-table-names=0
MDEV-33085 Tables T1 and t1 do not work well with ENGINE=CSV and lower-case-table-names=0
MDEV-33086 SHOW OPEN TABLES IN DB1 -- is case insensitive with lower-case-table-names=0
MDEV-33088 Cannot create triggers in the database `MYSQL`
MDEV-33103 LOCK TABLE t1 AS t2 -- alias is not case sensitive with lower-case-table-names=0
MDEV-33109 DROP DATABASE MYSQL -- does not drop SP with lower-case-table-names=0
MDEV-33110 HANDLER commands are case insensitive with lower-case-table-names=0
MDEV-33119 User is case insensitive in INFORMATION_SCHEMA.VIEWS
MDEV-33120 System log table names are case insensitive with lower-cast-table-names=0
- Removing the virtual function strnncoll() from MY_COLLATION_HANDLER
- Adding a wrapper function CHARSET_INFO::streq(), to compare
two strings for equality. For now it calls strnncoll() internally.
In the future it will turn into a virtual function.
- Adding new accent sensitive case insensitive collations:
- utf8mb4_general1400_as_ci
- utf8mb3_general1400_as_ci
They implement accent sensitive case insensitive comparison.
The weight of a character is equal to the code point of its
upper case variant. These collations use Unicode-14.0.0 casefolding data.
The result of
my_charset_utf8mb3_general1400_as_ci.strcoll()
is very close to the former
my_charset_utf8mb3_general_ci.strcasecmp()
There is only a difference in a couple dozen rare characters, because:
- the switch from "tolower" to "toupper" comparison, to make
utf8mb3_general1400_as_ci closer to utf8mb3_general_ci
- the switch from Unicode-3.0.0 to Unicode-14.0.0
This difference should be tolarable. See the list of affected
characters in the MDEV description.
Note, utf8mb4_general1400_as_ci correctly handles non-BMP characters!
Unlike utf8mb4_general_ci, it does not treat all BMP characters
as equal.
- Adding classes representing names of the file based database objects:
Lex_ident_db
Lex_ident_table
Lex_ident_trigger
Their comparison collation depends on the underlying
file system case sensitivity and on --lower-case-table-names
and can be either my_charset_bin or my_charset_utf8mb3_general1400_as_ci.
- Adding classes representing names of other database objects,
whose names have case insensitive comparison style,
using my_charset_utf8mb3_general1400_as_ci:
Lex_ident_column
Lex_ident_sys_var
Lex_ident_user_var
Lex_ident_sp_var
Lex_ident_ps
Lex_ident_i_s_table
Lex_ident_window
Lex_ident_func
Lex_ident_partition
Lex_ident_with_element
Lex_ident_rpl_filter
Lex_ident_master_info
Lex_ident_host
Lex_ident_locale
Lex_ident_plugin
Lex_ident_engine
Lex_ident_server
Lex_ident_savepoint
Lex_ident_charset
engine_option_value::Name
- All the mentioned Lex_ident_xxx classes implement a method streq():
if (ident1.streq(ident2))
do_equal();
This method works as a wrapper for CHARSET_INFO::streq().
- Changing a lot of "LEX_CSTRING name" to "Lex_ident_xxx name"
in class members and in function/method parameters.
- Replacing all calls like
system_charset_info->coll->strcasecmp(ident1, ident2)
to
ident1.streq(ident2)
- Taking advantage of the c++11 user defined literal operator
for LEX_CSTRING (see m_strings.h) and Lex_ident_xxx (see lex_ident.h)
data types. Use example:
const Lex_ident_column primary_key_name= "PRIMARY"_Lex_ident_column;
is now a shorter version of:
const Lex_ident_column primary_key_name=
Lex_ident_column({STRING_WITH_LEN("PRIMARY")});
Changing the format in error messages:
- ER_PACKAGE_ROUTINE_IN_SPEC_NOT_DEFINED_IN_BODY
- ER_PACKAGE_ROUTINE_FORWARD_DECLARATION_NOT_DEFINED
from
"Subroutine 'db.pkg.f1' ..."
to a more clear:
"FUNCTION `db.pkg.f1` ..."
"PROCEDURE `db.pkg.p1` ..."
The MDEV-29693 conflict resolution is from Monty, as well as is
a bug fix where ANALYZE TABLE wrongly built histograms for
single-column PRIMARY KEY.
Also includes a fix for safe_malloc error reporting.
Other things:
- Copied main.log_slow from 10.4 to avoid mtr issue
Disabled test:
- spider/bugfix.mdev_27239 because we started to get
+Error 1429 Unable to connect to foreign data source: localhost
-Error 1158 Got an error reading communication packets
- main.delayed
- Bug#54332 Deadlock with two connections doing LOCK TABLE+INSERT DELAYED
This part is disabled for now as it fails randomly with different
warnings/errors (no corruption).
Problem:
Under terms of MDEV-27490, we'll update Unicode version used
to compare identifiers to 14.0.0. Unlike in the old Unicode version,
in the new version a string can grow during lower-case. We cannot
perform check_db_name() inplace any more.
Change summary:
- Allocate memory to store lower-cased identifiers in memory root
- Removing check_db_name() performing both in-place lower-casing and validation
at the same time. Splitting it into two separate stages:
* creating a memory-root lower-cased copy of an identifier
(using new MEM_ROOT functions and Query_arena wrapper methods)
* performing validation on a constant string
(using Lex_ident_fs methods)
Implementation details:
- Adding a mysys helper function to allocate lower-cased strings on MEM_ROOT:
lex_string_casedn_root()
and a Query_arena wrappers for it:
make_ident_casedn()
make_ident_opt_casedn()
- Adding a Query_arena method to perform both MEM_ROOT lower-casing and
database name validation at the same time:
to_ident_db_internal_with_error()
This method is very close to the old (pre-11.3) check_db_name(),
but performs lower-casing to a newly allocated MEM_ROOT
memory (instead of performing lower-casing the original string in-place).
- Adding a Table_ident method which additionally handles derived table names:
to_ident_db_internal_with_error()
- Removing the old check_db_name()
This patch adds support for controlling of memory allocation
done by SP/PS that could happen on second and following executions.
As soon as SP or PS has been executed the first time its memory root
is marked as read only since no further memory allocation should
be performed on it. In case such allocation takes place it leads to
the assert hit for invariant that force no new memory allocations
takes place as soon as the SP/PS has been marked as read only.
The feature for control of memory allocation made on behalf SP/PS
is turned on when both debug build is on and the cmake option
-DWITH_PROTECT_STATEMENT_MEMROOT is set.
The reason for introduction of the new cmake option
-DWITH_PROTECT_STATEMENT_MEMROOT
to control memory allocation of second and following executions of
SP/PS is that for the current server implementation there are too many
places where such memory allocation takes place. As soon as all such
incorrect allocations be fixed the cmake option
-DWITH_PROTECT_STATEMENT_MEMROOT
can be removed and control of memory allocation made on second and
following executions can be turned on only for debug build. Before
every incorrect memory allocation be fixed it makes sense to guard
the checking of memory allocation on read only memory by extra cmake
option else we would get a lot of failing test on buildbot.
Moreover, fixing of all incorrect memory allocations could take pretty
long period of time, so for introducing the feature without necessary
to wait until all places throughout the source code be fixed it makes
sense to add the new cmake option.
Remove redundant delete_explain_query() calls in
sp_instr_set::exec_core(), sp_instr_set_row_field::exec_core(),
sp_instr_set_row_field_by_name::exec_core().
These calls are made before the SP instruction's tables are
"closed" by close_thread_tables() call.
When we call close_thread_tables() after that, we no longer
can collect engine's counter variables, as they use the data
structures that are located in the Explain Data Structures.
Also, these delete_explain_query() calls are redundant, as
sp_lex_keeper::reset_lex_and_exec_core() has another
delete_explain_query() call, which is located in the right
location after the close_thread_tables() call.
Introduced the new data member new_query_arena_is_set of the class sp_head.
This data member is used as a protection against double invocation of
the method restore_thd_mem_root that is called for restoration of the
current query arena. Previously, the data member sp_head::m_thd
is used for this goal but after support for re-compilation of a failed
stored routine statement has been added the data member sp_head::m_thd
can't be used for this goal. The reason is that on a statement re-compilation
after the method restore_thd_mem_root() is called the method
sp_head::add_instr() invoked to add a new instruction for just re-compiled
statement. The method sp_head::add_instr() de-references m_thd to access
the free_list data member. If m_thd was used as a guard against double
invocation it would result in a crash on dereferencing null pointer.
Fixed memory leakage taken place on execution of the statement
SHOW CREATE PACKAGE `pkg_name`
The memory leak was caused by implementation of sp_compile()
where a memory root for a stored routine was allocated but
a pointer to the new memory root wasn't passed to sp_package::create
for subsequent forwarding to the constructor of sp_package.
Instead, another one memory root was allocated and the pointer to
the original memory root was missed.
Re-designed a way by that Item_trigger_field objects are arranged in memory.
Item_trigger_field objects created on parsing a trigger's statement
is now stored in a per statement list. All lists of Item_trigger_field
objects created on parsing the whole trigger's body are organized
in the structure "list of lists". So, use binary cycle to iterate every
Item_trigger_field object created on parsing a trigger body.
To organize the data structure 'list of lists' the new data member
Item_trigger_field::next_trig_field_list
is introduced that links lists in this hierarchy structure.
This re-design is performed in order to avoid refences to already
deleted items on re-compilation of failed trigger's statememt.
Referencing to already deleted items could take place on re-parsing
a trigger's statement since every Item created for a statement
being re-parsed is deleted before the statement be re-parsed,
but deleted items are still referenced from sp_head. So, to avoid
access to dangling references a per statement list of Item_trigger_field
objects are cleared right after the current SP statement be cleaned up
and before re-parsing be started.
Added re-parsing of failed statements inside a stored routine.
General idea of the patch is to install an instance of the class
Reprepare_observer before executing a next SP instruction and
re-parse a statement of this SP instruction in case of
its execution failure.
To implement the described approach the class sp_lex_keeper
has been extended with the method validate_lex_and_exec_core()
that is just a wrapper around the method reset_lex_and_exec_core()
with additional setting/resetting an instance of the class
Reprepare_observer on each iteration of SP instruction
execution.
If reset_lex_and_exec_core() returns error and an instance
of the class Reprepare_observer is installed before running
a SP instruction then a number of attempts to re-run the SP
instruction is checked against a max. limit and in case it doesn't
reach the limit a statement for the failed SP instruction is re-parsed.
Re-parsing of a statement for the failed SP instruction is implemented
by the new method sp_le_inst::parse_expr() that prepends
a SP instruction's statement with the clause 'SELECT' and parse it.
Own SP instruction MEM_ROOT and a separate free_list is used for
parsing of a SP statement. On successful re-parsing of SP instruction's
statement the virtual methods adjust_sql_command() and
on_after_expr_parsing() of the class sp_lex_instr is called
to update the SP instruction state with a new data created
on parsing the statement.
Few words about reason for prepending a SP instruction's statement
with the clause 'SELECT' - this is required step to produce a valid
SQL statement, since for some SP instructions the instructions statement
is not a valid SQL statement. Wrapping such text into 'SELECT ( )'
produces a correct operator from SQL syntax point of view.
For those SP instructions that need to get access to ia LEX object
on execution, added storing of their original sql expressions inside
classes derived from the class sp_lex_instr.
A stored sql expression is returned by the abstract method
sp_lex_instr::get_expr_query
redefined in derived classes.
Since an expression constituting a SP instruction can be invalid
SQL statement in general case (not parseable statement), the virtual
method sp_lex_instr::get_query() is introduced to return a valid string
for a statement that corresponds to the given instruction.
Additionally, introduced the rule remember_start_opt in the grammar.
The new rule intended to get correct position of a current
token taking into attention the fact whether lookahead was done or not.
This is the prerequisite patch to change a signature of the virtual
method opt_move() in the base class sp_instr and classes derived from it.
The parameterized type of the instuctions list returned in the second
argument is changed from sp_instr to sp_instr_opt_meta since only
jump instructions are placed in this list on returning from
the method call.
This is the prerequisite patch to move the data member
LEX::trg_table_fields to the class sp_head and rename it as
m_trg_table_fields.
This data member is used for handling OLD/NEW pseudo-rows inside
a trigger body and in order to be able to re-parse a trigger body
the data member must be moved from the struct LEX to the class sp_head.
This is the prerequisite patch to move the sp_instr class and classes derived
from it into the files sp_instr.cc/sp_instr.h. The classes sp_lex_cursor and
sp_lex_keeper are also moved to the files files sp_instr.cc/sp_instr.h.
Additionally,
* all occurrences of macroses NULL, FALSE, TRUE are replaced
with the corresponding C++ keywords nullptr, false, true.
* the keyword 'override' is added in and the keyword 'virtual' is removed
from signatures of every virtual method implemented in classes derived
from the base class sp_instr.
* the keyword 'final' is added into declaration of the class sp_lex_keeper
since this class shouldn't have a derived class by design.
* the function cmp_rqp_locations is made static since it is not called
outside the file sp_instr.cc.
* the function subst_spvars() is moved into the file sp_instr.cc since this
function used only by the method sp_instr_stmt::execute
- Moving the code from a public function trim_whitespaces()
to the class Lex_cstring as methods. This code may
be useful in other contexts, and also this code becomes
visible inside sql_class.h
- Adding a helper method THD::strmake_lex_cstring_trim_whitespaces()
- Unifying the way how CREATE PROCEDURE/CREATE FUNCTION and
CREATE PACKAGE/CREATE PACKAGE BODY work:
a) Now CREATE PACKAGE/CREATE PACKAGE BODY also calls
Lex->sphead->set_body_start() to remember the cpp body start inside
an sp_head member.
b) adding a "const char *cpp_body_end" parameter to
sp_head::set_stmt_end().
These changes made it possible to reuse sp_head::set_stmt_end() inside
LEX::create_package_finalize() and remove the duplucate code.
- Renaming sp_head::m_body_begin to m_cpp_body_begin and adding a comment
to make it clear that this member is used only during parsing, and
points to a fragment inside the cpp buffer.
- Changed sp_head::set_body_start() and sp_head::set_stmt_end()
to skip the calls related to "body_utf8" in cases when m_parent is not NULL.
A non-NULL m_parent means that we're inside a package routine.
"body_utf8" in such case belongs not to the current sphead itself,
but to parent (the package) sphead.
So an sphead instance of a package routine should neither initialize,
nor finalize, nor change in any other ways the "body_utf8" related
members of Lex_input_stream, and should not take over or copy "body_utf8"
data from Lex_input_stream to "this".
- agressively -> aggressively
- exising -> existing
- occured -> occurred
- releated -> related
- seperated -> separated
- sucess -> success
- use use -> use
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.
Since commit d7d3ad698a, "hard" kill is
required to interrupt debug sync waits.
Affected the following tests:
- galera_var_retry_autocommit,
- galera_bf_abort_at_after_statement
- galera_parallel_apply_3nodes
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
