The patch for bug mdev-9937 actually did not fix the problem
of name resolution for tables used in views referred in queries
with WITH clauses. This fix corrects the patch.
mysqld maintains a list of TABLE objects for all temporary
tables created within a session in THD. Here each table is
represented by a TABLE object.
A query referencing a particular temporary table for more
than once, however, failed with ER_CANT_REOPEN_TABLE error
because a TABLE_SHARE was allocate together with the TABLE,
so temporary tables always had only one TABLE per TABLE_SHARE.
This patch lift this restriction by separating TABLE and
TABLE_SHARE objects and storing TABLE_SHAREs for temporary
tables in a list in THD, and TABLEs in a list within their
respective TABLE_SHAREs.
When the specification of a WITH table referred to a view
that used a based table with the same name as the WITH table
the server went into an infinite loop because it erroneously
resolved the reference to the base table as the reference to
the WITH table.
With tables used in a view cannot be searched for beyond the
scope the view.
* only copy args[0] to args[2] after fix_fields (when all item
substitutions have already happened)
* change QT_ITEM_FUNC_NULLIF_TO_CASE (that allows to print NULLIF
as CASE) to QT_ITEM_ORIGINAL_FUNC_NULLIF (that prohibits it).
So that NULLIF-to-CASE is allowed by default and only disabled
explicitly for SHOW VIEW|FUNCTION|PROCEDURE and mysql_make_view.
By default it is allowed (in particular in error messages and
debug output, that can happen anytime before or after optimizer).
Problem:
Procedure which uses stack of views first executed without most deep view.
It fails but one view cached (as well as whole procedure).
Then simultaniusely create the second view we lack and execute the procedure.
In the beginning of procedure execution the view is not yet created so
procedure used as it was cached (cache was not invalidated).
But by the time we are trying to use most deep view it is already created.
The problem with the view is that thd->select_number (first view was not parsed) so second view will get the same number.
The fix is in keeping the thd->select_number correct even if we use cached views.
In the proposed solution (to keep it simple) counter can be bigger then should but it should not create problem because numbers are still unique and situation is very rare.
- Part 3: Adding mem_root to push_back() and push_front()
Other things:
- Added THD as an argument to some partition functions.
- Added memory overflow checking for XML tag's in read_xml()
- Changed ER(ER_...) to ER_THD(thd, ER_...) when thd was known or if there was many calls to current_thd in the same function.
- Changed ER(ER_..) to ER_THD_OR_DEFAULT(current_thd, ER...) in some places where current_thd is not necessary defined.
- Removing calls to current_thd when we have access to thd
Part of this is optimization (not calling current_thd when not needed),
but part is bug fixing for error condition when current_thd is not defined
(For example on startup and end of mysqld)
Notable renames done as otherwise a lot of functions would have to be changed:
- In JOIN structure renamed:
examined_rows -> join_examined_rows
record_count -> join_record_count
- In Field, renamed new_field() to make_new_field()
Other things:
- Added DBUG_ASSERT(thd == tmp_thd) in Item_singlerow_subselect() just to be safe.
- Removed old 'tab' prefix in JOIN_TAB::save_explain_data() and use members directly
- Added 'thd' as argument to a few functions to avoid calling current_thd.
Call mysql_derived_reinit() if we are reusing view.
This is needed as during a previous error condition the view may not have been reset
sql/sql_derived.cc:
More DBUG_PRINT
Always reset merged_for_insert (no reason to not do that)
sql/sql_derived.h:
Added prototype
sql/sql_insert.cc:
More DBUG_PRINT
Added DBUG_ASSERT
sql/sql_view.cc:
Call mysql_derived_reinit() if we are reusing view.
This is needed as during a previous error condition the view may not have been reset
sql/table.cc:
More DBUG_PRINT
The reason for the failure was a bug in an include file on debian that causes 'struct stat'
to have different sized depending on the environment.
This patch fixes so that we always include my_global.h or my_config.h before we include any other files.
Other things:
- Removed #include <my_global.h> in some include files; Better to always do this at the top level to have as few
"always-include-this-file-first' files as possible.
- Removed usage of some include files that where already included by my_global.h or by other files.
client/mysql_plugin.c:
Use my_global.h first
client/mysqlslap.c:
Remove duplicated include files
extra/comp_err.c:
Remove duplicated include files
include/m_string.h:
Remove duplicated include files
include/maria.h:
Remove duplicated include files
libmysqld/emb_qcache.cc:
Use my_global.h first
plugin/semisync/semisync.h:
Use my_pthread.h first
sql/datadict.cc:
Use my_global.h first
sql/debug_sync.cc:
Use my_global.h first
sql/derror.cc:
Use my_global.h first
sql/des_key_file.cc:
Use my_global.h first
sql/discover.cc:
Use my_global.h first
sql/event_data_objects.cc:
Use my_global.h first
sql/event_db_repository.cc:
Use my_global.h first
sql/event_parse_data.cc:
Use my_global.h first
sql/event_queue.cc:
Use my_global.h first
sql/event_scheduler.cc:
Use my_global.h first
sql/events.cc:
Use my_global.h first
sql/field.cc:
Use my_global.h first
Remove duplicated include files
sql/field_conv.cc:
Use my_global.h first
sql/filesort.cc:
Use my_global.h first
Remove duplicated include files
sql/gstream.cc:
Use my_global.h first
sql/ha_ndbcluster.cc:
Use my_global.h first
sql/ha_ndbcluster_binlog.cc:
Use my_global.h first
sql/ha_ndbcluster_cond.cc:
Use my_global.h first
sql/ha_partition.cc:
Use my_global.h first
sql/handler.cc:
Use my_global.h first
sql/hash_filo.cc:
Use my_global.h first
sql/hostname.cc:
Use my_global.h first
sql/init.cc:
Use my_global.h first
sql/item.cc:
Use my_global.h first
sql/item_buff.cc:
Use my_global.h first
sql/item_cmpfunc.cc:
Use my_global.h first
sql/item_create.cc:
Use my_global.h first
sql/item_geofunc.cc:
Use my_global.h first
sql/item_inetfunc.cc:
Use my_global.h first
sql/item_row.cc:
Use my_global.h first
sql/item_strfunc.cc:
Use my_global.h first
sql/item_subselect.cc:
Use my_global.h first
sql/item_sum.cc:
Use my_global.h first
sql/item_timefunc.cc:
Use my_global.h first
sql/item_xmlfunc.cc:
Use my_global.h first
sql/key.cc:
Use my_global.h first
sql/lock.cc:
Use my_global.h first
sql/log.cc:
Use my_global.h first
sql/log_event.cc:
Use my_global.h first
sql/log_event_old.cc:
Use my_global.h first
sql/mf_iocache.cc:
Use my_global.h first
sql/mysql_install_db.cc:
Remove duplicated include files
sql/mysqld.cc:
Remove duplicated include files
sql/net_serv.cc:
Remove duplicated include files
sql/opt_range.cc:
Use my_global.h first
sql/opt_subselect.cc:
Use my_global.h first
sql/opt_sum.cc:
Use my_global.h first
sql/parse_file.cc:
Use my_global.h first
sql/partition_info.cc:
Use my_global.h first
sql/procedure.cc:
Use my_global.h first
sql/protocol.cc:
Use my_global.h first
sql/records.cc:
Use my_global.h first
sql/records.h:
Don't include my_global.h
Better to do this at the upper level
sql/repl_failsafe.cc:
Use my_global.h first
sql/rpl_filter.cc:
Use my_global.h first
sql/rpl_gtid.cc:
Use my_global.h first
sql/rpl_handler.cc:
Use my_global.h first
sql/rpl_injector.cc:
Use my_global.h first
sql/rpl_record.cc:
Use my_global.h first
sql/rpl_record_old.cc:
Use my_global.h first
sql/rpl_reporting.cc:
Use my_global.h first
sql/rpl_rli.cc:
Use my_global.h first
sql/rpl_tblmap.cc:
Use my_global.h first
sql/rpl_utility.cc:
Use my_global.h first
sql/set_var.cc:
Added comment
sql/slave.cc:
Use my_global.h first
sql/sp.cc:
Use my_global.h first
sql/sp_cache.cc:
Use my_global.h first
sql/sp_head.cc:
Use my_global.h first
sql/sp_pcontext.cc:
Use my_global.h first
sql/sp_rcontext.cc:
Use my_global.h first
sql/spatial.cc:
Use my_global.h first
sql/sql_acl.cc:
Use my_global.h first
sql/sql_admin.cc:
Use my_global.h first
sql/sql_analyse.cc:
Use my_global.h first
sql/sql_audit.cc:
Use my_global.h first
sql/sql_base.cc:
Use my_global.h first
sql/sql_binlog.cc:
Use my_global.h first
sql/sql_bootstrap.cc:
Use my_global.h first
Use my_global.h first
sql/sql_cache.cc:
Use my_global.h first
sql/sql_class.cc:
Use my_global.h first
sql/sql_client.cc:
Use my_global.h first
sql/sql_connect.cc:
Use my_global.h first
sql/sql_crypt.cc:
Use my_global.h first
sql/sql_cursor.cc:
Use my_global.h first
sql/sql_db.cc:
Use my_global.h first
sql/sql_delete.cc:
Use my_global.h first
sql/sql_derived.cc:
Use my_global.h first
sql/sql_do.cc:
Use my_global.h first
sql/sql_error.cc:
Use my_global.h first
sql/sql_explain.cc:
Use my_global.h first
sql/sql_expression_cache.cc:
Use my_global.h first
sql/sql_handler.cc:
Use my_global.h first
sql/sql_help.cc:
Use my_global.h first
sql/sql_insert.cc:
Use my_global.h first
sql/sql_lex.cc:
Use my_global.h first
sql/sql_load.cc:
Use my_global.h first
sql/sql_locale.cc:
Use my_global.h first
sql/sql_manager.cc:
Use my_global.h first
sql/sql_parse.cc:
Use my_global.h first
sql/sql_partition.cc:
Use my_global.h first
sql/sql_plugin.cc:
Added comment
sql/sql_prepare.cc:
Use my_global.h first
sql/sql_priv.h:
Added error if we use this before including my_global.h
This check is here becasue so many files includes sql_priv.h first.
sql/sql_profile.cc:
Use my_global.h first
sql/sql_reload.cc:
Use my_global.h first
sql/sql_rename.cc:
Use my_global.h first
sql/sql_repl.cc:
Use my_global.h first
sql/sql_select.cc:
Use my_global.h first
sql/sql_servers.cc:
Use my_global.h first
sql/sql_show.cc:
Added comment
sql/sql_signal.cc:
Use my_global.h first
sql/sql_statistics.cc:
Use my_global.h first
sql/sql_table.cc:
Use my_global.h first
sql/sql_tablespace.cc:
Use my_global.h first
sql/sql_test.cc:
Use my_global.h first
sql/sql_time.cc:
Use my_global.h first
sql/sql_trigger.cc:
Use my_global.h first
sql/sql_udf.cc:
Use my_global.h first
sql/sql_union.cc:
Use my_global.h first
sql/sql_update.cc:
Use my_global.h first
sql/sql_view.cc:
Use my_global.h first
sql/sys_vars.cc:
Added comment
sql/table.cc:
Use my_global.h first
sql/thr_malloc.cc:
Use my_global.h first
sql/transaction.cc:
Use my_global.h first
sql/uniques.cc:
Use my_global.h first
sql/unireg.cc:
Use my_global.h first
sql/unireg.h:
Removed inclusion of my_global.h
storage/archive/ha_archive.cc:
Added comment
storage/blackhole/ha_blackhole.cc:
Use my_global.h first
storage/csv/ha_tina.cc:
Use my_global.h first
storage/csv/transparent_file.cc:
Use my_global.h first
storage/federated/ha_federated.cc:
Use my_global.h first
storage/federatedx/federatedx_io.cc:
Use my_global.h first
storage/federatedx/federatedx_io_mysql.cc:
Use my_global.h first
storage/federatedx/federatedx_io_null.cc:
Use my_global.h first
storage/federatedx/federatedx_txn.cc:
Use my_global.h first
storage/heap/ha_heap.cc:
Use my_global.h first
storage/innobase/handler/handler0alter.cc:
Use my_global.h first
storage/maria/ha_maria.cc:
Use my_global.h first
storage/maria/unittest/ma_maria_log_cleanup.c:
Remove duplicated include files
storage/maria/unittest/test_file.c:
Added comment
storage/myisam/ha_myisam.cc:
Move sql_plugin.h first as this includes my_global.h
storage/myisammrg/ha_myisammrg.cc:
Use my_global.h first
storage/oqgraph/oqgraph_thunk.cc:
Use my_config.h and my_global.h first
One could not include my_global.h before oqgraph_thunk.h (don't know why)
storage/spider/ha_spider.cc:
Use my_global.h first
storage/spider/hs_client/config.cpp:
Use my_global.h first
storage/spider/hs_client/escape.cpp:
Use my_global.h first
storage/spider/hs_client/fatal.cpp:
Use my_global.h first
storage/spider/hs_client/hstcpcli.cpp:
Use my_global.h first
storage/spider/hs_client/socket.cpp:
Use my_global.h first
storage/spider/hs_client/string_util.cpp:
Use my_global.h first
storage/spider/spd_conn.cc:
Use my_global.h first
storage/spider/spd_copy_tables.cc:
Use my_global.h first
storage/spider/spd_db_conn.cc:
Use my_global.h first
storage/spider/spd_db_handlersocket.cc:
Use my_global.h first
storage/spider/spd_db_mysql.cc:
Use my_global.h first
storage/spider/spd_db_oracle.cc:
Use my_global.h first
storage/spider/spd_direct_sql.cc:
Use my_global.h first
storage/spider/spd_i_s.cc:
Use my_global.h first
storage/spider/spd_malloc.cc:
Use my_global.h first
storage/spider/spd_param.cc:
Use my_global.h first
storage/spider/spd_ping_table.cc:
Use my_global.h first
storage/spider/spd_sys_table.cc:
Use my_global.h first
storage/spider/spd_table.cc:
Use my_global.h first
storage/spider/spd_trx.cc:
Use my_global.h first
storage/xtradb/handler/handler0alter.cc:
Use my_global.h first
storage/xtradb/handler/i_s.cc:
Use my_global.h first
~40% bugfixed(*) applied
~40$ bugfixed reverted (incorrect or we're not buggy)
~20% bugfixed applied, despite us being not buggy
(*) only changes in the server code, e.g. not cmakefiles
THD::thd->activate_stmt_arena_if_needed() should be used to temporary activating statement arena instead of direct usage of THD::set_n_backup_active_arena() because possible such scenario:
1) func1 saves current arena and activates copy1 of statement arena
2) func2 saves copy1 of statement arena setup by func1 and activates copy2
3) some changes made for copy 2
4) func2 stores changed copy2 back to statenet arena and activates copy1
5) func1 store unchanged copy1 back to statemnt arena (rewrite changed copy 2 so changes become lost) and activates arena which was before.
THD::thd->activate_stmt_arena_if_needed() should be used to temporary activating statement arena instead of direct usage of THD::set_n_backup_active_arena() because possible such scenario:
1) func1 saves current arena and activates copy1 of statement arena
2) func2 saves copy1 of statement arena setup by func1 and activates copy2
3) some changes made for copy 2
4) func2 stores changed copy2 back to statenet arena and activates copy1
5) func1 store unchanged copy1 back to statemnt arena (rewrite changed copy 2 so changes become lost) and activates arena which was before.
sql/sp.cc:
don't split "user@host" string in db_load_routine, because the caller needs to
generate it from user and host. instead pass user and host directly into db_load_routine
sql/sql_parse.cc:
1. REVOKE ALL doesn't need invoker.
2. make sp_process_definer() reusable
sql/sql_trigger.cc:
don't duplicate the code from sp_process_definer(), reuse it
sql/sql_view.cc:
don't duplicate the code from sp_process_definer(), reuse it
- Port grant_explain_non_select.{test,result} from mysql-5.6
- Per Sanja's hint, fix mysql_make_view() to take into
account that EXPLAIN now is not necessarily EXPLAIN SELECT.
includes:
* remove some remnants of "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
* introduce LOCK_share, now LOCK_ha_data is strictly for engines
* rea_create_table() always creates .par file (even in "frm-only" mode)
* fix a 5.6 bug, temp file leak on dummy ALTER TABLE
- temporary tables now works
- mysql-system_tables updated to not use temporary tables
- PASSWORD() function fixed
- Support for STATS_AUTO_RECALC, STATS_PERSISTENT and STATS_SAMPLE_PAGES table options
Syntax modified to allow statements:
ALTER TABLE ADD/DROP COLUMN
ALTER TABLE ADD/DROP INDEX
ALTER TABLE ADD/DROP FOREIGN KEY
ALTER TABLE ADD/DROP PARTITION
ALTER TABLE CHANGE COLUMN
ALTER TABLE MODIFY COLUMN
DROP INDEX
to have IF (NOT) EXISTS options.
Appropriate implementations added to mysql_alter_table().
per-file comments:
mysql-test/r/alter_table.result
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
test result updated.
mysql-test/r/fulltext.result
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
mysql-test/r/partition.result
test result updated.
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
mysql-test/t/alter_table.test
tests added.
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
mysql-test/t/fulltext.test
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
tests added.
mysql-test/t/partition.test
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
tests added.
sql/field.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists field added.
sql/field.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists field added.
sql/partition_info.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
has_unique_name made public.
sql/sp_head.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
sql/sql_class.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists inited.
sql/sql_class.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
create_if_not_exists inited.
sql/sql_lex.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists inited.
sql/sql_lex.h
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists inited.
sql/sql_parse.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists inited.
sql/sql_table.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
handle_if_exists_options() added.
it's called in mysql_alter_table().
sql/sql_trigger.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists instead of drop_if_exists.
sql/sql_view.cc
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
check_exists instead of drop_if_exists.
sql/sql_yacc.yy
MDEV-318 IF (NOT) EXIST clauses for ALTER TABLE (MWL #252).
sintax modified.
AVAILABLE MEMORY IS TOO LOW
Analysis:
---------
In function "mysql_make_view", "table->view" is initialized
after parsing(using File_parser::parse) the view definition.
If "::parse" function fails then control is moved to label
"err:". Here we have assert (table->view == thd->lex).
This assert fails if "::parse" function fails, as
table->view is not initialized yet.
File_parser::parse fails if data being parsed is incorrect/
corrupted or when memory allocation fails. In this scenario
its failing because of failure in memory allocation.
Fix:
---------
In case of failure in function "File_parser::parse", moving
to label "err:" is incorrect. Modified code to move
to label "end:".
and small collateral changes
mysql-test/lib/My/Test.pm:
somehow with "print" we get truncated writes sometimes
mysql-test/suite/perfschema/r/digest_table_full.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/dml_handler.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/information_schema.result:
host table is not ported over yet
mysql-test/suite/perfschema/r/nesting.result:
this differs, because we don't rewrite general log queries, and multi-statement
packets are logged as a one entry. this result file is identical to what mysql-5.6.5
produces with the --log-raw option.
mysql-test/suite/perfschema/r/relaylog.result:
MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB.
mysql-test/suite/perfschema/r/server_init.result:
MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup.
mysql-test/suite/perfschema/r/stage_mdl_global.result:
this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not
do that, and this causes useless mutex locks and waits.
mysql-test/suite/perfschema/r/statement_digest.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_consumers.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/perfschema/r/statement_digest_long_query.result:
md5 hashes of statement digests differ, because yacc token codes are different in mariadb
mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result:
will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged
mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result:
will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged
As part of derived tables redesign, values for VIEW_ALGORITHM_MERGE and VIEW_ALGORITHM_TMPTABLE have changed from (former values 1 rsp 2 , new values 5 rsp 9).
This lead to the problem that views, created with version 5.2 or earlier would not work in all situations (e.g "SHOW CREATE VIEW"), or with mysqldump.
The fix is to restore backward compatibility for the from file, and convert algorithm={1,2} in the frm to {5,9} when reading .frm from disk, and store backward compatible values when writing from to disk.
Also allow processing correct processing for "invalid" .frms created with MariaDB 5.3/5.5 GA releases (where algorithm stored in memory matched the one stored in frm).
ENOUGH - CONCAT() HACKS. ALSO WRONG
ERROR MESSAGE WHILE TRYING TO CREATE
A VIEW ON A NON EXISTING DATABASE
PROBLEM:
The first part of the problem is concluded as not a
bug, as 'concat' is not a reserved word and it is
completely valid to create a view with the name
'concat'.
The second issue is, while trying to create a view on
a non existing database, we are not giving a proper error
message.
FIX:
We have added a check for the database existence while
trying to create a view. This check would give an error
as 'unknown database' when the database does not exist.
This patch is a backport of the patch for Bug#13601606
mysql-test/r/view.result:
Added test case result of Bug#12626844
mysql-test/t/view.test:
Added test case for Bug#12626844
sql/sql_view.cc:
Added a check for database existence in mysql_create_view
https://mariadb.atlassian.net/browse/MDEV-28
This task implements a new clause LIMIT ROWS EXAMINED <num>
as an extention to the ANSI LIMIT clause. This extension
allows to limit the number of rows and/or keys a query
would access (read and/or write) during query execution.
client/mysqltest.cc:
Free mutex after usage (fixes valgrind warnings in embedded server)
mysql-test/include/gis_keys.inc:
Fixed failure in innodb.gis_test
mysql-test/r/gis.result:
Updated result
mysql-test/suite/innodb/r/innodb_gis.result:
Updated results
mysql-test/suite/innodb/t/innodb_bug38231.test:
Added handling of timeouts (happend on some servers in buildbot)
mysql-test/suite/innodb_plugin/r/innodb_gis.result:
Updated results
mysql-test/suite/innodb_plugin/t/innodb.test:
Use error names instead of numbers
mysql-test/suite/innodb_plugin/t/innodb_misc1.test:
This test requires utf8
mysql-test/suite/innodb_plugin/t/innodb_mysql.test:
This test requires Xtradb
sql/sql_base.cc:
Don't print table names for placeholders.
sql/sql_show.cc:
Temporary fix:
Save and restore db and table_name in mysqld_show_create (to get rid of valgrind warning)
A better solution that needs to be investgated is to not change these fields in mysql_derived_prepare()
sql/sql_view.cc:
Fixed valgrind warning
storage/xtradb/handler/ha_innodb.cc:
Don't access THD directly
The problem was that when we have single row subquery with no rows
Item_cache(es) which represent result row was not null and being
requested via element_index() returned random value.
The fix is setting all Item_cache(es) in NULL before executing the
query (reset() method) which guaranty NULL value of whole query
or its elements requested in any way if no rows was found.
set_null() method was added to Item_cache to guaranty correct NULL
value in case of reseting the cache.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
Cleanup: Changed procedure type from a int/char to an enum for easier to manage and debug code.
mysql-test/r/trigger.result:
Test that mysql.proc is not used as part of creating or using a trigger.
mysql-test/t/trigger.test:
Test that mysql.proc is not used as part of creating or using a trigger.
sql/sp.cc:
The main bug fix is to not look up triggers in mysql.proc; This is done by ignoreing type == TYPE_ENUM_TRIGGER in sp_add_used_routine()
Cleanup: Changed procedure type from a int/char to an enum.
sql/sp.h:
Cleanup: Changed procedure type from a int/char to an enum.
sql/sp_head.h:
Cleanup: Changed procedure type from a int/char to an enum.
sql/sql_db.cc:
Fix include order
sql/sql_lex.cc:
Fix include order
sql/sql_parse.cc:
Cleanup: Changed procedure type from a int/char to an enum.
sql/sql_show.cc:
Fix include order
sql/sql_view.cc:
Fix include order
1 - If a user had SHOW VIEW and SELECT privileges on a view and
this view was referencing another view, EXPLAIN SELECT on the outer
view (that the user had privileges on) could reveal the structure
of the underlying "inner" view as well as the number of rows in
the underlying tables, even if the user had privileges on none of
these referenced objects.
This happened because we used DEFINER's UID ("SUID") not just for
the view given in EXPLAIN, but also when checking privileges on
the underlying views (where we should use the UID of the EXPLAIN's
INVOKER instead).
We no longer run the EXPLAIN SUID (with DEFINER's privileges).
This prevents a possible exploit and makes permissions more
orthogonal.
2 - EXPLAIN SELECT would reveal a view's structure even if the user
did not have SHOW VIEW privileges for that view, as long as they
had SELECT privilege on the underlying tables.
Instead of requiring both SHOW VIEW privilege on a view and SELECT
privilege on all underlying tables, we were checking for presence
of either of them.
We now explicitly require SHOW VIEW and SELECT privileges on
the view we run EXPLAIN SELECT on, as well as all its
underlying views. We also require SELECT on all relevant
tables.
mysql-test/r/view_grant.result:
add extensive tests to illustrate desired behavior and
prevent regressions (as always).
mysql-test/t/view_grant.test:
add extensive tests to illustrate desired behavior and
prevent regressions (as always).
sql/sql_view.cc:
We no longer run the EXPLAIN SUID (with DEFINER's privileges).
To achieve this, we use a temporary, SUID-less TABLE_LIST for
the views while checking privileges.
In case of two views with subqueries it is dificult to decide about order of injected ORDER BY clauses.
A simple solution is just prohibit ORDER BY injection if there is other order by.
mysql-test/r/view.result:
New test added, old test changed.
mysql-test/t/view.test:
New test aded.
sql/share/errmsg.txt:
new warning added.
sql/sql_view.cc:
Inject ORDER BY only if there is no other one.
Warning about ignoring ORDER BY in this case for EXPLAIN EXTENDED.
Resolved all conflicts, bad merges and fixed a few minor bugs in the code.
Commented out the queries from multi_update, view, subselect_sj, func_str,
derived_view, view_grant that failed either with crashes in ps-protocol or
with wrong results.
The failures are clear indications of some bugs in the code and these bugs
are to be fixed.
@ mysql-test/r/ctype_latin1.result
@ mysql-test/r/ctype_utf8.result
@ mysql-test/t/ctype_latin1.test
@ mysql-test/t/ctype_utf8.test
Adding tests
@ sql/mysqld.h
@ sql/item.cc
@ sql/sql_parse.cc
@ sql/sql_view.cc
Refactoring (thanks to Guilhem for the idea):
Item_string::print() was hard to understand because of the different
QT_ constants: in "query_type==QT_x", QT_x is explicitely included
but the other two QT_ are implicitely excluded. The combinations
with '||' and '&&' make this even harder.
- logic is now more "explicit" by changing QT_ constants to a bitmap of flags:
QT_ORDINARY: no change,
QT_IS -> QT_TO_SYSTEM_CHARSET | QT_WITHOUT_INTRODUCERS,
QT_EXPLAIN -> QT_TO_SYSTEM_CHARSET
(QT_EXPLAIN was introduced in the first version of the Bug#57341 patch)
- Item_string::print() is rewritten using those flags
Bugfix itself:
When QT_TO_SYSTEM_CHARSET is used alone (with no QT_WITHOUT_INTRODUCERS),
we print string literals as follows:
- display introducers if they were in the original query
- print ASCII characters as is
- print non-ASCII characters using hex-escape
Note: as "EXPLAIN" output is only for human readability purposes
and does not need to be a pasrable SQL, so using hex-escape is Ok.
ErrConvString class perfectly suites for hex escaping purposes.
Changed some String.ptr() -> String.c_ptr() for String that are not guaranteed to end with \0
Removed some c_ptr() usage from parameters to functions that takes ptr & length
Use preallocate buffers to avoid calling malloc() for most operations.
sql/event_db_repository.cc:
alias is now a String
sql/event_scheduler.cc:
c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
sql/events.cc:
c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
c_ptr -> ptr() as function takes ptr & length
sql/field.cc:
alias is now a String
sql/field.h:
alias is now a String
sql/ha_partition.cc:
alias is now a String
sql/handler.cc:
alias is now a String
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/item.cc:
Store error parameter in separarte buffer to ensure correct error message
sql/item_func.cc:
ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/item_sum.h:
Use my_strtod() instead of my_atof() to not have to make string \0 terminated
sql/lock.cc:
alias is now a String
sql/log.cc:
c_ptr() -> ptr() as function takes ptr & length
sql/log_event.cc:
c_ptr_quick() -> ptr() as we only want to get the pointer to String buffer
sql/opt_range.cc:
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/opt_table_elimination.cc:
alias is now a String
sql/set_var.cc:
ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
c_ptr() -> ptr() as function takes ptr & length
Simplify some code.
sql/sp.cc:
c_ptr() -> ptr() as function takes ptr & length
sql/sp_rcontext.cc:
alias is now a String
sql/sql_base.cc:
alias is now a String.
Here we win a realloc() for most alias usage.
sql/sql_class.cc:
Use size descriptor for printf() to avoid accessing bytes outside of buffer
sql/sql_insert.cc:
Change allocation of TABLE as it's now contains a String
_ptr() -> ptr() as function takes ptr & length
sql/sql_load.cc:
Use preallocate buffers to avoid calling malloc() for most operations.
sql/sql_parse.cc:
Use c_ptr_safe() to ensure string is \0 terminated.
sql/sql_plugin.cc:
c_ptr_quick() -> ptr() as function takes ptr & length
sql/sql_select.cc:
alias is now a String
sql/sql_show.cc:
alias is now a String
sql/sql_string.h:
Added move() function to change who owns the string (owner does the free)
sql/sql_table.cc:
alias is now a String
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
sql/sql_test.cc:
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
alias is now a String
sql/sql_trigger.cc:
c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
Use field->init() to setup pointers to alias.
sql/sql_update.cc:
alias is now a String
sql/sql_view.cc:
ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/sql_yacc.yy:
r() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/table.cc:
alias is now a String
sql/table.h:
alias is now a String
storage/federatedx/ha_federatedx.cc:
Remove extra 1 byte alloc that is automaticly done by strmake()
Ensure that error message ends with \0
storage/maria/ha_maria.cc:
alias is now a String
storage/myisam/ha_myisam.cc:
alias is now a String
- Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria"
- Fixed wrong assert in Aria
Now need to merge with latest xtradb before pushing
sql/ha_partition.cc:
Ensure that m_ordered_rec_buffer is not freed before close.
sql/mysqld.cc:
Changed to use opt_stack_trace instead of opt_pstack.
Removed references to pstack
sql/partition_element.h:
Ensure that connect_string is initialized
storage/maria/ma_key_recover.c:
Fixed wrong assert
INVOKER-security view access check wrong".
When privilege checks were done for tables used from an
INVOKER-security view which in its turn was used from
a DEFINER-security view connection's active security
context was incorrectly used instead of security context
with privileges of the second view's creator.
This meant that users which had enough rights to access
the DEFINER-security view and as result were supposed to
be able successfully access it were unable to do so in
cases when they didn't have privileges on underlying tables
of the INVOKER-security view.
This problem was caused by the fact that for INVOKER-security
views TABLE_LIST::security_ctx member for underlying tables
were set to 0 even in cases when particular view was used from
another DEFINER-security view. This meant that when checks of
privileges on these underlying tables was done in
setup_tables_and_check_access() active connection security
context was used instead of context corresponding to the
creator of caller view.
This fix addresses the problem by ensuring that underlying
tables of an INVOKER-security view inherit security context
from the view and thus correct security context is used for
privilege checks on underlying tables in cases when such view
is used from another view with DEFINER-security.
mysql-test/r/view_grant.result:
Added coverage for various combinations of DEFINER and
INVOKER-security views, including test for bug #58499
"DEFINER-security view selecting from INVOKER-security
view access check wrong".
mysql-test/t/view_grant.test:
Added coverage for various combinations of DEFINER and
INVOKER-security views, including test for bug #58499
"DEFINER-security view selecting from INVOKER-security
view access check wrong".
sql/sql_view.cc:
When opening a non-suid view ensure that its underlying
tables will get the same security context as use for
checking privileges on the view, i.e. security context
of view invoker. This context can be different from the
security context which is currently active for connection
in cases when this non-suid view is used from a view with
suid security. Inheriting security context in such situation
allows correctly apply privileges of creator of suid view
in checks for tables of non-suid view (since in this
situation creator/definer of suid view serves as invoker
for non-suid view).
--Bug#52157 various crashes and assertions with multi-table update, stored function
--Bug#54475 improper error handling causes cascading crashing failures in innodb/ndb
--Bug#57703 create view cause Assertion failed: 0, file .\item_subselect.cc, line 846
--Bug#57352 valgrind warnings when creating view
--Recently discovered problem when a nested materialized derived table is used
before being populated and it leads to incorrect result
We have several modes when we should disable subquery evaluation.
The reasons for disabling are different. It could be
uselessness of the evaluation as in case of 'CREATE VIEW'
or 'PREPARE stmt', or we should disable subquery evaluation
if tables are not locked yet as it happens in bug#54475, or
too early evaluation of subqueries can lead to wrong result
as it happened in Bug#19077.
Main problem is that if subquery items are treated as const
they are evaluated in ::fix_fields(), ::fix_length_and_dec()
of the parental items as a lot of these methods have
Item::val_...() calls inside.
We have to make subqueries non-const to prevent unnecessary
subquery evaluation. At the moment we have different methods
for this. Here is a list of these modes:
1. PREPARE stmt;
We use UNCACHEABLE_PREPARE flag.
It is set during parsing in sql_parse.cc, mysql_new_select() for
each SELECT_LEX object and cleared at the end of PREPARE in
sql_prepare.cc, init_stmt_after_parse(). If this flag is set
subquery becomes non-const and evaluation does not happen.
2. CREATE|ALTER VIEW, SHOW CREATE VIEW, I_S tables which
process FRM files
We use LEX::view_prepare_mode field. We set it before
view preparation and check this flag in
::fix_fields(), ::fix_length_and_dec().
Some bugs are fixed using this approach,
some are not(Bug#57352, Bug#57703). The problem here is
that we have a lot of ::fix_fields(), ::fix_length_and_dec()
where we use Item::val_...() calls for const items.
3. Derived tables with subquery = wrong result(Bug19077)
The reason of this bug is too early subquery evaluation.
It was fixed by adding Item::with_subselect field
The check of this field in appropriate places prevents
const item evaluation if the item have subquery.
The fix for Bug19077 fixes only the problem with
convert_constant_item() function and does not cover
other places(::fix_fields(), ::fix_length_and_dec() again)
where subqueries could be evaluated.
Example:
CREATE TABLE t1 (i INT, j BIGINT);
INSERT INTO t1 VALUES (1, 2), (2, 2), (3, 2);
SELECT * FROM (SELECT MIN(i) FROM t1
WHERE j = SUBSTRING('12', (SELECT * FROM (SELECT MIN(j) FROM t1) t2))) t3;
DROP TABLE t1;
4. Derived tables with subquery where subquery
is evaluated before table locking(Bug#54475, Bug#52157)
Suggested solution is following:
-Introduce new field LEX::context_analysis_only with the following
possible flags:
#define CONTEXT_ANALYSIS_ONLY_PREPARE 1
#define CONTEXT_ANALYSIS_ONLY_VIEW 2
#define CONTEXT_ANALYSIS_ONLY_DERIVED 4
-Set/clean these flags when we perform
context analysis operation
-Item_subselect::const_item() returns
result depending on LEX::context_analysis_only.
If context_analysis_only is set then we return
FALSE that means that subquery is non-const.
As all subquery types are wrapped by Item_subselect
it allow as to make subquery non-const when
it's necessary.
mysql-test/r/derived.result:
test case
mysql-test/r/multi_update.result:
test case
mysql-test/r/view.result:
test case
mysql-test/suite/innodb/r/innodb_multi_update.result:
test case
mysql-test/suite/innodb/t/innodb_multi_update.test:
test case
mysql-test/suite/innodb_plugin/r/innodb_multi_update.result:
test case
mysql-test/suite/innodb_plugin/t/innodb_multi_update.test:
test case
mysql-test/t/derived.test:
test case
mysql-test/t/multi_update.test:
test case
mysql-test/t/view.test:
test case
sql/item.cc:
--removed unnecessary code
sql/item_cmpfunc.cc:
--removed unnecessary checks
--THD::is_context_analysis_only() is replaced with LEX::is_ps_or_view_context_analysis()
sql/item_func.cc:
--refactored context analysis checks
sql/item_row.cc:
--removed unnecessary checks
sql/item_subselect.cc:
--removed unnecessary code
--added DBUG_ASSERT into Item_subselect::exec()
which asserts that subquery execution can not happen
if LEX::context_analysis_only is set, i.e. at context
analysis stage.
--Item_subselect::const_item()
Return FALSE if LEX::context_analysis_only is set.
It prevents subquery evaluation in ::fix_fields &
::fix_length_and_dec at context analysis stage.
sql/item_subselect.h:
--removed unnecessary code
sql/mysql_priv.h:
--Added new set of flags.
sql/sql_class.h:
--removed unnecessary code
sql/sql_derived.cc:
--added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_lex.cc:
--init LEX::context_analysis_only field
sql/sql_lex.h:
--New LEX::context_analysis_only field
sql/sql_parse.cc:
--removed unnecessary code
sql/sql_prepare.cc:
--removed unnecessary code
--added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_select.cc:
--refactored context analysis checks
sql/sql_show.cc:
--added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_view.cc:
--added LEX::context_analysis_only analysis intialization/cleanup
bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ
LOCK" and bug #54673 "It takes too long to get readlock for
'FLUSH TABLES WITH READ LOCK'".
The first bug manifested itself as a deadlock which occurred
when a connection, which had some table open through HANDLER
statement, tried to update some data through DML statement
while another connection tried to execute FLUSH TABLES WITH
READ LOCK concurrently.
What happened was that FTWRL in the second connection managed
to perform first step of GRL acquisition and thus blocked all
upcoming DML. After that it started to wait for table open
through HANDLER statement to be flushed. When the first connection
tried to execute DML it has started to wait for GRL/the second
connection creating deadlock.
The second bug manifested itself as starvation of FLUSH TABLES
WITH READ LOCK statements in cases when there was a constant
stream of concurrent DML statements (in two or more
connections).
This has happened because requests for protection against GRL
which were acquired by DML statements were ignoring presence of
pending GRL and thus the latter was starved.
This patch solves both these problems by re-implementing GRL
using metadata locks.
Similar to the old implementation acquisition of GRL in new
implementation is two-step. During the first step we block
all concurrent DML and DDL statements by acquiring global S
metadata lock (each DML and DDL statement acquires global IX
lock for its duration). During the second step we block commits
by acquiring global S lock in COMMIT namespace (commit code
acquires global IX lock in this namespace).
Note that unlike in old implementation acquisition of
protection against GRL in DML and DDL is semi-automatic.
We assume that any statement which should be blocked by GRL
will either open and acquires write-lock on tables or acquires
metadata locks on objects it is going to modify. For any such
statement global IX metadata lock is automatically acquired
for its duration.
The first problem is solved because waits for GRL become
visible to deadlock detector in metadata locking subsystem
and thus deadlocks like one in the first bug become impossible.
The second problem is solved because global S locks which
are used for GRL implementation are given preference over
IX locks which are acquired by concurrent DML (and we can
switch to fair scheduling in future if needed).
Important change:
FTWRL/GRL no longer blocks DML and DDL on temporary tables.
Before this patch behavior was not consistent in this respect:
in some cases DML/DDL statements on temporary tables were
blocked while in others they were not. Since the main use cases
for FTWRL are various forms of backups and temporary tables are
not preserved during backups we have opted for consistently
allowing DML/DDL on temporary tables during FTWRL/GRL.
Important change:
This patch changes thread state names which are used when
DML/DDL of FTWRL is waiting for global read lock. It is now
either "Waiting for global read lock" or "Waiting for commit
lock" depending on the stage on which FTWRL is.
Incompatible change:
To solve deadlock in events code which was exposed by this
patch we have to replace LOCK_event_metadata mutex with
metadata locks on events. As result we have to prohibit
DDL on events under LOCK TABLES.
This patch also adds extensive test coverage for interaction
of DML/DDL and FTWRL.
Performance of new and old global read lock implementations
in sysbench tests were compared. There were no significant
difference between new and old implementations.
mysql-test/include/check_ftwrl_compatible.inc:
Added helper script which allows to check that a statement is
compatible with FLUSH TABLES WITH READ LOCK.
mysql-test/include/check_ftwrl_incompatible.inc:
Added helper script which allows to check that a statement is
incompatible with FLUSH TABLES WITH READ LOCK.
mysql-test/include/handler.inc:
Adjusted test case to the fact that now DROP TABLE closes
open HANDLERs for the table to be dropped before checking
if there active FTWRL in this connection.
mysql-test/include/wait_show_condition.inc:
Fixed small error in the timeout message. The correct name
of variable used as parameter for this script is "$condition"
and not "$wait_condition".
mysql-test/r/delayed.result:
Added test coverage for scenario which triggered assert in
metadata locking subsystem.
mysql-test/r/events_2.result:
Updated test results after prohibiting event DDL operations
under LOCK TABLES.
mysql-test/r/flush.result:
Added test coverage for bug #57006 "Deadlock between HANDLER
and FLUSH TABLES WITH READ LOCK".
mysql-test/r/flush_read_lock.result:
Added test coverage for various aspects of FLUSH TABLES WITH
READ LOCK functionality.
mysql-test/r/flush_read_lock_kill.result:
Adjusted test case after replacing custom global read lock
implementation with one based on metadata locks. Use new
debug_sync point. Do not disable concurrent inserts as now
InnoDB we always use InnoDB table.
mysql-test/r/handler_innodb.result:
Adjusted test case to the fact that now DROP TABLE closes
open HANDLERs for the table to be dropped before checking
if there active FTWRL in this connection.
mysql-test/r/handler_myisam.result:
Adjusted test case to the fact that now DROP TABLE closes
open HANDLERs for the table to be dropped before checking
if there active FTWRL in this connection.
mysql-test/r/mdl_sync.result:
Adjusted test case after replacing custom global read lock
implementation with one based on metadata locks. Replaced
usage of GRL-specific debug_sync's with appropriate sync
points in MDL subsystem.
mysql-test/suite/perfschema/r/dml_setup_instruments.result:
Updated test results after removing global
COND_global_read_lock condition variable.
mysql-test/suite/perfschema/r/func_file_io.result:
Ensure that this test doesn't affect subsequent tests.
At the end of its execution enable back P_S instrumentation
which this test disables at some point.
mysql-test/suite/perfschema/r/func_mutex.result:
Ensure that this test doesn't affect subsequent tests.
At the end of its execution enable back P_S instrumentation
which this test disables at some point.
mysql-test/suite/perfschema/r/global_read_lock.result:
Adjusted test case to take into account that new GRL
implementation is based on MDL.
mysql-test/suite/perfschema/r/server_init.result:
Adjusted test case after replacing custom global read
lock implementation with one based on MDL and replacing
LOCK_event_metadata mutex with metadata lock.
mysql-test/suite/perfschema/t/func_file_io.test:
Ensure that this test doesn't affect subsequent tests.
At the end of its execution enable back P_S instrumentation
which this test disables at some point.
mysql-test/suite/perfschema/t/func_mutex.test:
Ensure that this test doesn't affect subsequent tests.
At the end of its execution enable back P_S instrumentation
which this test disables at some point.
mysql-test/suite/perfschema/t/global_read_lock.test:
Adjusted test case to take into account that new GRL
implementation is based on MDL.
mysql-test/suite/perfschema/t/server_init.test:
Adjusted test case after replacing custom global read
lock implementation with one based on MDL and replacing
LOCK_event_metadata mutex with metadata lock.
mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result:
Updated test results after prohibiting event DDL under
LOCK TABLES.
mysql-test/t/delayed.test:
Added test coverage for scenario which triggered assert in
metadata locking subsystem.
mysql-test/t/events_2.test:
Updated test case after prohibiting event DDL operations
under LOCK TABLES.
mysql-test/t/flush.test:
Added test coverage for bug #57006 "Deadlock between HANDLER
and FLUSH TABLES WITH READ LOCK".
mysql-test/t/flush_block_commit.test:
Adjusted test case after changing thread state name which
is used when COMMIT waits for FLUSH TABLES WITH READ LOCK
from "Waiting for release of readlock" to "Waiting for commit
lock".
mysql-test/t/flush_block_commit_notembedded.test:
Adjusted test case after changing thread state name which is
used when DML waits for FLUSH TABLES WITH READ LOCK. Now we
use "Waiting for global read lock" in this case.
mysql-test/t/flush_read_lock.test:
Added test coverage for various aspects of FLUSH TABLES WITH
READ LOCK functionality.
mysql-test/t/flush_read_lock_kill-master.opt:
We no longer need to use make_global_read_lock_block_commit_loop
debug tag in this test. Instead we rely on an appropriate
debug_sync point in MDL code.
mysql-test/t/flush_read_lock_kill.test:
Adjusted test case after replacing custom global read lock
implementation with one based on metadata locks. Use new
debug_sync point. Do not disable concurrent inserts as now
InnoDB we always use InnoDB table.
mysql-test/t/lock_multi.test:
Adjusted test case after changing thread state names which
are used when DML or DDL waits for FLUSH TABLES WITH READ
LOCK to "Waiting for global read lock".
mysql-test/t/mdl_sync.test:
Adjusted test case after replacing custom global read lock
implementation with one based on metadata locks. Replaced
usage of GRL-specific debug_sync's with appropriate sync
points in MDL subsystem. Updated thread state names which
are used when DDL waits for FTWRL.
mysql-test/t/trigger_notembedded.test:
Adjusted test case after changing thread state names which
are used when DML or DDL waits for FLUSH TABLES WITH READ
LOCK to "Waiting for global read lock".
sql/event_data_objects.cc:
Removed Event_queue_element::status/last_executed_changed
members and Event_queue_element::update_timing_fields()
method. We no longer use this class for updating mysql.events
once event is chosen for execution. Accesses to instances of
this class in scheduler thread require protection by
Event_queue::LOCK_event_queue mutex and we try to avoid
updating table while holding this lock.
sql/event_data_objects.h:
Removed Event_queue_element::status/last_executed_changed
members and Event_queue_element::update_timing_fields()
method. We no longer use this class for updating mysql.events
once event is chosen for execution. Accesses to instances of
this class in scheduler thread require protection by
Event_queue::LOCK_event_queue mutex and we try to avoid
updating table while holding this lock.
sql/event_db_repository.cc:
- Changed Event_db_repository methods to not release all
metadata locks once they are done updating mysql.events
table. This allows to keep metadata lock protecting
against GRL and lock protecting particular event around
until corresponding DDL statement is written to the binary
log.
- Removed logic for conditional update of "status" and
"last_executed" fields from update_timing_fields_for_event()
method. In the only case when this method is called now
"last_executed" is always modified and tracking change
of "status" is too much hassle.
sql/event_db_repository.h:
Removed logic for conditional update of "status" and
"last_executed" fields from Event_db_repository::
update_timing_fields_for_event() method.
In the only case when this method is called now "last_executed"
is always modified and tracking change of "status" field is
too much hassle.
sql/event_queue.cc:
Changed event scheduler code not to update mysql.events
table while holding Event_queue::LOCK_event_queue mutex.
Doing so led to a deadlock with a new GRL implementation.
This deadlock didn't occur with old implementation due to
fact that code acquiring protection against GRL ignored
pending GRL requests (which lead to GRL starvation).
One of goals of new implementation is to disallow GRL
starvation and so we have to solve problem with this
deadlock in a different way.
sql/events.cc:
Changed methods of Events class to acquire protection
against GRL while perfoming DDL statement and keep it
until statement is written to the binary log.
Unfortunately this step together with new GRL implementation
exposed deadlock involving Events::LOCK_event_metadata
and GRL. To solve it Events::LOCK_event_metadata mutex was
replaced with a metadata lock on event. As a side-effect
events DDL has to be prohibited under LOCK TABLES even in
cases when mysql.events table was explicitly locked for
write.
sql/events.h:
Replaced Events::LOCK_event_metadata mutex with a metadata
lock on event.
sql/ha_ndbcluster.cc:
Updated code after replacing custom global read lock
implementation with one based on MDL. Since MDL subsystem
should now be able to detect deadlocks involving metadata
locks and GRL there is no need for special handling of
active GRL.
sql/handler.cc:
Replaced custom implementation of global read lock with
one based on metadata locks. Consequently when doing
commit instead of calling method of Global_read_lock
class to acquire protection against GRL we simply acquire
IX in COMMIT namespace.
sql/lock.cc:
Replaced custom implementation of global read lock with
one based on metadata locks. This step allows to expose
wait for GRL to deadlock detector of MDL subsystem and
thus succesfully resolve deadlocks similar to one behind
bug #57006 "Deadlock between HANDLER and FLUSH TABLES
WITH READ LOCK". It also solves problem with GRL starvation
described in bug #54673 "It takes too long to get readlock
for 'FLUSH TABLES WITH READ LOCK'" since metadata locks used
by GRL give preference to FTWRL statement instead of DML
statements (if needed in future this can be changed to
fair scheduling).
Similar to old implementation of acquisition of GRL is
two-step. During the first step we block all concurrent
DML and DDL statements by acquiring global S metadata lock
(each DML and DDL statement acquires global IX lock for
its duration). During the second step we block commits by
acquiring global S lock in COMMIT namespace (commit code
acquires global IX lock in this namespace).
Note that unlike in old implementation acquisition of
protection against GRL in DML and DDL is semi-automatic.
We assume that any statement which should be blocked by GRL
will either open and acquires write-lock on tables or acquires
metadata locks on objects it is going to modify. For any such
statement global IX metadata lock is automatically acquired
for its duration.
To support this change:
- Global_read_lock::lock/unlock_global_read_lock and
make_global_read_lock_block_commit methods were changed
accordingly.
- Global_read_lock::wait_if_global_read_lock() and
start_waiting_global_read_lock() methods were dropped.
It is now responsibility of code acquiring metadata locks
opening tables to acquire protection against GRL by
explicitly taking global IX lock with statement duration.
- Global variables, mutex and condition variable used by
old implementation was removed.
- lock_routine_name() was changed to use statement duration for
its global IX lock. It was also renamed to lock_object_name()
as it now also used to take metadata locks on events.
- Global_read_lock::set_explicit_lock_duration() was added which
allows not to release locks used for GRL when leaving prelocked
mode.
sql/lock.h:
- Renamed lock_routine_name() to lock_object_name() and changed
its signature to allow its usage for events.
- Removed broadcast_refresh() function. It is no longer needed
with new GRL implementation.
sql/log_event.cc:
Release metadata locks with statement duration at the end
of processing legacy event for LOAD DATA. This ensures that
replication thread processing such event properly releases
its protection against global read lock.
sql/mdl.cc:
Changed MDL subsystem to support new MDL-based implementation
of global read lock.
Added COMMIT and EVENTS namespaces for metadata locks. Changed
thread state name for GLOBAL namespace to "Waiting for global
read lock".
Optimized MDL_map::find_or_insert() method to avoid taking
m_mutex mutex when looking up MDL_lock objects for GLOBAL
or COMMIT namespaces. We keep pre-created MDL_lock objects
for these namespaces around and simply return pointers to
these global objects when needed.
Changed MDL_lock/MDL_scoped_lock to properly handle
notification of insert delayed handler threads when FTWRL
takes global S lock.
Introduced concept of lock duration. In addition to locks with
transaction duration which work in the way which is similar to
how locks worked before (i.e. they are released at the end of
transaction), locks with statement and explicit duration were
introduced.
Locks with statement duration are automatically released at the
end of statement. Locks with explicit duration require explicit
release and obsolete concept of transactional sentinel.
* Changed MDL_request and MDL_ticket classes to support notion
of duration.
* Changed MDL_context to keep locks with different duration in
different lists. Changed code handling ticket list to take
this into account.
* Changed methods responsible for releasing locks to take into
account duration of tickets. Particularly public
MDL_context::release_lock() method now only can release
tickets with explicit duration (there is still internal
method which allows to specify duration). To release locks
with statement or transaction duration one have to use
release_statement/transactional_locks() methods.
* Concept of savepoint for MDL subsystem now has to take into
account locks with statement duration. Consequently
MDL_savepoint class was introduced and methods working with
savepoints were updated accordingly.
* Added methods which allow to set duration for one or all
locks in the context.
sql/mdl.h:
Changed MDL subsystem to support new MDL-based implementation
of global read lock.
Added COMMIT and EVENTS namespaces for metadata locks.
Introduced concept of lock duration. In addition to locks with
transaction duration which work in the way which is similar to
how locks worked before (i.e. they are released at the end of
transaction), locks with statement and explicit duration were
introduced.
Locks with statement duration are automatically released at the
end of statement. Locks with explicit duration require explicit
release and obsolete concept of transactional sentinel.
* Changed MDL_request and MDL_ticket classes to support notion
of duration.
* Changed MDL_context to keep locks with different duration in
different lists. Changed code handling ticket list to take
this into account.
* Changed methods responsible for releasing locks to take into
account duration of tickets. Particularly public
MDL_context::release_lock() method now only can release
tickets with explicit duration (there is still internal
method which allows to specify duration). To release locks
with statement or transaction duration one have to use
release_statement/transactional_locks() methods.
* Concept of savepoint for MDL subsystem now has to take into
account locks with statement duration. Consequently
MDL_savepoint class was introduced and methods working with
savepoints were updated accordingly.
* Added methods which allow to set duration for one or all
locks in the context.
sql/mysqld.cc:
Removed global mutex and condition variables which were used
by old implementation of GRL.
Also we no longer need to initialize Events::LOCK_event_metadata
mutex as it was replaced with metadata locks on events.
sql/mysqld.h:
Removed global variable, mutex and condition variables which
were used by old implementation of GRL.
sql/rpl_rli.cc:
When slave thread closes tables which were open for handling
of RBR events ensure that it releases global IX lock which
was acquired as protection against GRL.
sql/sp.cc:
Adjusted code to the new signature of lock_object/routine_name(),
to the fact that one now needs specify duration of lock when
initializing MDL_request and to the fact that savepoints for MDL
subsystem are now represented by MDL_savepoint class.
sql/sp_head.cc:
Ensure that statements in stored procedures release statement
metadata locks and thus release their protectiong against GRL
in proper moment in time.
Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request.
sql/sql_admin.cc:
Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request.
sql/sql_base.cc:
- Implemented support for new approach to acquiring protection
against global read lock. We no longer acquire such protection
explicitly on the basis of statement flags. Instead we always
rely on code which is responsible for acquiring metadata locks
on object to be changed acquiring this protection. This is
achieved by acquiring global IX metadata lock with statement
duration. Code doing this also responsible for checking that
current connection has no active GRL by calling an
Global_read_lock::can_acquire_protection() method.
Changed code in open_table() and lock_table_names()
accordingly.
Note that as result of this change DDL and DML on temporary
tables is always compatible with GRL (before it was
incompatible in some cases and compatible in other cases).
- To speed-up code acquiring protection against GRL introduced
m_has_protection_against_grl member in Open_table_context
class. It indicates that protection was already acquired
sometime during open_tables() execution and new attempts
can be skipped.
- Thanks to new GRL implementation calls to broadcast_refresh()
became unnecessary and were removed.
- Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request and to the fact that
savepoints for MDL subsystem are now represented by
MDL_savepoint class.
sql/sql_base.h:
Adjusted code to the fact that savepoints for MDL subsystem are
now represented by MDL_savepoint class.
Also introduced Open_table_context::m_has_protection_against_grl
member which allows to avoid acquiring protection against GRL
while opening tables if such protection was already acquired.
sql/sql_class.cc:
Changed THD::leave_locked_tables_mode() after transactional
sentinel for metadata locks was obsoleted by introduction of
locks with explicit duration.
sql/sql_class.h:
- Adjusted code to the fact that savepoints for MDL subsystem
are now represented by MDL_savepoint class.
- Changed Global_read_lock class according to changes in
global read lock implementation:
* wait_if_global_read_lock and start_waiting_global_read_lock
are now gone. Instead code needing protection against GRL
has to acquire global IX metadata lock with statement
duration itself. To help it new can_acquire_protection()
was introduced. Also as result of the above change
m_protection_count member is gone too.
* Added m_mdl_blocks_commits_lock member to store metadata
lock blocking commits.
* Adjusted code to the fact that concept of transactional
sentinel was obsoleted by concept of lock duration.
- Removed CF_PROTECT_AGAINST_GRL flag as it is no longer
necessary. New GRL implementation acquires protection
against global read lock automagically when statement
acquires metadata locks on tables or other objects it
is going to change.
sql/sql_db.cc:
Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request.
sql/sql_handler.cc:
Removed call to broadcast_refresh() function. It is no longer
needed with new GRL implementation.
Adjusted code after introducing duration concept for metadata
locks. Particularly to the fact transactional sentinel was
replaced with explicit duration.
sql/sql_handler.h:
Renamed mysql_ha_move_tickets_after_trans_sentinel() to
mysql_ha_set_explicit_lock_duration() after transactional
sentinel was obsoleted by locks with explicit duration.
sql/sql_insert.cc:
Adjusted code handling delaying inserts after switching to
new GRL implementation. Now connection thread initiating
delayed insert has to acquire global IX lock in addition
to metadata lock on table being inserted into. This IX lock
protects against GRL and similarly to SW lock on table being
inserted into has to be passed to handler thread in order to
avoid deadlocks.
sql/sql_lex.cc:
LEX::protect_against_global_read_lock member is no longer
necessary since protection against GRL is automatically
taken by code acquiring metadata locks/opening tables.
sql/sql_lex.h:
LEX::protect_against_global_read_lock member is no longer
necessary since protection against GRL is automatically
taken by code acquiring metadata locks/opening tables.
sql/sql_parse.cc:
- Implemented support for new approach to acquiring protection
against global read lock. We no longer acquire such protection
explicitly on the basis of statement flags. Instead we always
rely on code which is responsible for acquiring metadata locks
on object to be changed acquiring this protection. This is
achieved by acquiring global IX metadata lock with statement
duration. This lock is automatically released at the end of
statement execution.
- Changed implementation of CREATE/DROP PROCEDURE/FUNCTION not
to release metadata locks and thus protection against of GRL
in the middle of statement execution.
- Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request and to the fact that
savepoints for MDL subsystem are now represented by
MDL_savepoint class.
sql/sql_prepare.cc:
Adjusted code to the to the fact that savepoints for MDL
subsystem are now represented by MDL_savepoint class.
sql/sql_rename.cc:
With new GRL implementation there is no need to explicitly
acquire protection against GRL before renaming tables.
This happens automatically in code which acquires metadata
locks on tables being renamed.
sql/sql_show.cc:
Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request and to the fact that
savepoints for MDL subsystem are now represented by
MDL_savepoint class.
sql/sql_table.cc:
- With new GRL implementation there is no need to explicitly
acquire protection against GRL before dropping tables.
This happens automatically in code which acquires metadata
locks on tables being dropped.
- Changed mysql_alter_table() not to release lock on new table
name explicitly and to rely on automatic release of locks
at the end of statement instead. This was necessary since
now MDL_context::release_lock() is supported only for locks
for explicit duration.
sql/sql_trigger.cc:
With new GRL implementation there is no need to explicitly
acquire protection against GRL before changing table triggers.
This happens automatically in code which acquires metadata
locks on tables which triggers are to be changed.
sql/sql_update.cc:
Fix bug exposed by GRL testing. During prepare phase acquire
only S metadata locks instead of SW locks to keep prepare of
multi-UPDATE compatible with concurrent LOCK TABLES WRITE
and global read lock.
sql/sql_view.cc:
With new GRL implementation there is no need to explicitly
acquire protection against GRL before creating view.
This happens automatically in code which acquires metadata
lock on view to be created.
sql/sql_yacc.yy:
LEX::protect_against_global_read_lock member is no longer
necessary since protection against GRL is automatically
taken by code acquiring metadata locks/opening tables.
sql/table.cc:
Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request.
sql/table.h:
Adjusted code to the fact that one now needs specify duration
of lock when initializing MDL_request.
sql/transaction.cc:
Replaced custom implementation of global read lock with
one based on metadata locks. Consequently when doing
commit instead of calling method of Global_read_lock
class to acquire protection against GRL we simply acquire
IX in COMMIT namespace.
Also adjusted code to the fact that MDL savepoint is now
represented by MDL_savepoint class.
Fall back to use ALTER TABLE for engines that doesn't support REPAIR when doing repair for upgrade.
Nicer output from mysql_upgrade and mysql_check
Updated all arrays that used NAME_LEN to use SAFE_NAME_LEN to ensure that we don't break things accidently as names can now have a #mysql50# prefix.
client/mysql_upgrade.c:
If we are using verbose, also run mysqlcheck in verbose mode.
client/mysqlcheck.c:
Add more information if running in verbose mode
Print 'Needs upgrade' instead of complex error if table needs to be upgraded
Don't write connect information if verbose is not 2 or above
mysql-test/r/drop.result:
Updated test and results as we now support full table names
mysql-test/r/grant.result:
Now you get a correct error message if using #mysql with paths
mysql-test/r/show_check.result:
Update results as table names can temporarly be bigger than NAME_LEN (during upgrade)
mysql-test/r/upgrade.result:
Test upgrade for long table names.
mysql-test/suite/funcs_1/r/is_tables_is.result:
Updated old test result (had note been updated in a while)
mysql-test/t/drop.test:
Updated test and results as we now support full table names
mysql-test/t/grant.test:
Now you get a correct error message if using #mysql with paths
mysql-test/t/upgrade.test:
Test upgrade for long table names.
sql/ha_partition.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/item.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/log_event.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/mysql_priv.h:
Added SAFE_NAME_LEN
sql/rpl_filter.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sp.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sp_head.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_acl.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_base.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_connect.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_parse.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_prepare.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_select.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_show.cc:
NAME_LEN -> SAFE_NAME_LEN
Enlarge table names for SHOW TABLES to also include optional #mysql50#
sql/sql_table.cc:
Fall back to use ALTER TABLE for engines that doesn't support REPAIR when doing repair for upgrade.
sql/sql_trigger.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_udf.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/sql_view.cc:
NAME_LEN -> SAFE_NAME_LEN
sql/table.cc:
Fixed check_table_name() to not count #mysql50# as part of name
If #mysql50# is part of the name, don't allow path characters in name.
'CREATE TABLE IF NOT EXISTS ... SELECT' behaviour
BUG#47132, BUG#47442, BUG49494, BUG#23992 and BUG#48814 will disappear
automatically after the this patch.
BUG#55617 is fixed by this patch too.
This is the 5.5 part.
It implements:
- 'CREATE TABLE IF NOT EXISTS ... SELECT' statement will not insert
anything and binlog anything if the table already exists.
It only generate a warning that table already exists.
- A couple of test cases for the behavior changing.
Remove acquisition of LOCK_open around file system operations,
since such operations are now protected by metadata locks.
Rework table discovery algorithm to not require LOCK_open.
No new tests added since all MDL locking operations are covered
in lock.test and mdl_sync.test, and as long as these tests
pass despite the increased concurrency, consistency must be
unaffected.
mysql-test/t/disabled.def:
Disable NDB tests due to Bug#55799.
sql/datadict.cc:
No longer necessary to protect ha_create_table() with
LOCK_open. Serial execution is now ensured by metadata
locks.
sql/ha_ndbcluster.cc:
Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.cc:
Do not manipulate with LOCK_open in cluster code.
sql/ha_ndbcluster_binlog.h:
Update function signature.
sql/handler.cc:
Implement ha_check_if_table_exists().
@todo: some engines provide ha_table_exists_in_engine()
handlerton call, for those we perhaps shouldn't
call ha_discover(), to be more efficient.
Since currently it's only NDB, postpone till
integration with NDB.
sql/handler.h:
Declare ha_check_if_table_exists() function.
sql/mdl.cc:
Remove an obsolete comment.
sql/sql_base.cc:
Update to a new signature of close_cached_tables():
from now on we always call it without LOCK_open.
Update comments.
Remove get_table_share_with_create(), we should
not attempt to create a table under LOCK_open.
Introduce get_table_share_with_discover() instead,
which would request a back off action if the table
exists in engine.
Remove acquisition of LOCK_open for
data dictionary operations, such as check_if_table_exists().
Do not use get_table_share_with_create/discover for views,
where it's not needed.
Make tdc_remove_table() optionally acquire LOCK_open
to simplify usage of this function.
Use the right mutex in the partitioning code when
manipulating with thd->open_tables.
sql/sql_base.h:
Update signatures of changes functions.
sql/sql_insert.cc:
Do not wrap quick_rm_table() with LOCK_open acquisition,
this is unnecessary.
sql/sql_parse.cc:
Update to the new calling convention of tdc_remove_table().
Update to the new signature of close_cached_tables().
Update comments.
sql/sql_rename.cc:
Update to the new calling convention of tdc_remove_table().
Remove acquisition of LOCK_open around filesystem
operations.
sql/sql_show.cc:
Remove get_trigger_table_impl().
Do not acquire LOCK_open for a dirty read of the trigger
file.
sql/sql_table.cc:
Do not acquire LOCK_open for filesystem operations.
sql/sql_trigger.cc:
Do not require LOCK_open for trigger file I/O.
sql/sql_truncate.cc:
Update to the new signature of tdc_remove_table().
sql/sql_view.cc:
Do not require LOCK_open for view I/O.
Use tdc_remove_table() to expel view share.
Update comments.
sql/sys_vars.cc:
Update to the new signature of close_cached_tables().
DATABASE with open HANDLER"
Remove LOCK_create_db, database name locks, and use metadata locks instead.
This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based
deadlock detector in MDL, and paves the way for a safe, deadlock-free
implementation of RENAME DATABASE.
Database DDL statements will now take exclusive metadata locks on
the database name, while table/view/routine DDL statements take
intention exclusive locks on the database name. This prevents race
conditions between database DDL and table/view/routine DDL.
(e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE)
By adding database name locks, this patch implements
WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and
WL#4985 "DDL locking: namespace/hierarchical locks".
The patch also changes code to use init_one_table() where appropriate.
The new lock_table_names() function requires TABLE_LIST::db_length to
be set correctly, and this is taken care of by init_one_table().
This patch also adds a simple template to help work with
the mysys HASH data structure.
Most of the patch was written by Konstantin Osipov.
