This bug happened because the function find_field_in_view formed
autogenerated names of view columns without a possibility to roll
them back. In some situation it could cause memory misuses reported
by valgrind or even crashes.
When a view/derived table is converted from merged to materialized the
items from the used_item lists are substituted for items referring to
the fields of the result of the materialization. The problem appeared
with queries employing natural joins. Since the resolution of a natural
join was performed only once the used_item list formed at the second
execution of the query lacked the references to the fields that were
used only in the equality predicates generated for the natural join.
mysql-test/suite/innodb/t/group_commit_crash.test:
remove autoincrement to avoid rbr being used for insert ... select
mysql-test/suite/innodb/t/group_commit_crash_no_optimize_thread.test:
remove autoincrement to avoid rbr being used for insert ... select
mysys/my_addr_resolve.c:
a pointer to a buffer is returned to the caller -> the buffer cannot be on the stack
mysys/stacktrace.c:
my_vsnprintf() is ok here, in 5.5
- Don't call update_virtual_fields() if table->vfield is not set
- Don't prealloc memory for in open_tables() as this is very seldom used.
sql/records.cc:
Don't call update_virtual_fields() if table->vfield is not set
sql/sql_base.cc:
Don't prealloc memory for in open_tables() as this is very seldom used.
Don't call update_virtual_fields() if table->vfield is not set
sql/sql_delete.cc:
Don't call update_virtual_fields() if table->vfield is not set
sql/sql_handler.cc:
Don't call update_virtual_fields() if table->vfield is not set
sql/sql_join_cache.cc:
Don't call update_virtual_fields() if table->vfield is not set
Move some frequent values to local variables
sql/sql_table.cc:
Don't call update_virtual_fields() if table->vfield is not set
sql/sql_update.cc:
Don't call update_virtual_fields() if table->vfield is not set
sql/table.cc:
Assert if update_virtual_fields is called with wrong parameters
- The bug would show up
- when using PS (so that we get re-execution)
- the left_expr of the subquery is a reference to viewname.column_name, so that it crashes
when one tries to use it without having called fix_fields for it.
- when using SJ-Materialization, which makes use of sj_subq_pred->left_expr expression
- The fix is to have setup_conds() fix sj_subq_pred->left_expr for semi-join nests it finds.
Fixed memory leaks and compiler warnings in ha_sphinx.cc
Added HA_MUST_USE_TABLE_CONDITION_PUSHDOWN so that an engine can force index condition to be used
mysql-test/suite/sphinx/sphinx.result:
Added testing of pushdown conditions and sphinx status variables.
mysql-test/suite/sphinx/sphinx.test:
Added testing of pushdown conditions and sphinx status variables.
mysql-test/suite/sphinx/suite.pm:
Print version number if sphinx version is too old.
sql/handler.h:
Added HA_MUST_USE_TABLE_CONDITION_PUSHDOWN so that an engine can force index condition to be used
sql/sql_base.cc:
Added 'thd' argument to check_unused() to be able to set 'entry->in_use' if we call handler->extra().
This was needed as sphinx (and possible other storage engines) assumes that 'in_use' is set if handler functions are called.
sql/sql_select.cc:
Test if handler is forcing pushdown condition to be used.
storage/sphinx/ha_sphinx.cc:
Updated to version 2.0.4
Fixed memory leaks and compiler warnings.
storage/sphinx/ha_sphinx.h:
Updated to version 2.0.4
storage/sphinx/snippets_udf.cc:
Updated to version 2.0.4
Fixed wrong mutex order bug in Aria when flush_log_for_bitmap() was called when table is not yet marked for change.
include/my_base.h:
Added flag that table is opened only for status
mysql-test/r/myisam-big.result:
Test case for lp:925377
mysql-test/t/myisam-big.test:
Test case for lp:925377
sql/sql_base.cc:
If thd->version == 0 (happens only when we are opening a table that is flushed under MYSQL_LOCK_IGNORE_FLUSH), open the table in HA_OPEN_FOR_STATUS mode
storage/maria/ma_bitmap.c:
Fixed wrong mutex order bug in Aria when flush_log_for_bitmap() was called when table is not yet marked for change.
storage/maria/ma_dbug.c:
Ignore last_version <= 1 as these are either flushed or only opened for status
storage/maria/ma_open.c:
Use last_version=1 as a marker that table was opened with HA_OPEN_FOR_STATUS.
In this case we just open a new version of the table in read only mode.
storage/myisam/mi_create.c:
Update prototype
storage/myisam/mi_dbug.c:
Ignore last_version <= 1 as these are either flushed or only opened for status
storage/myisam/mi_open.c:
Use last_version=1 as a marker that table was opened with HA_OPEN_FOR_STATUS.
If HA_OPEN_FOR_STATUS is used, we will not assert if there is an old not-to-be-used version of the table existing.
In this case we just open a new version of the table in read only mode.
storage/myisam/myisamdef.h:
Updated prototype
This also fixes a (not likely) crashing bug when forcing a thread that was doing a table lock to re-open it's files, for example by creating a trigger.
mysys/thr_lock.c:
Added more checking to find wrong locks.
Removed one, not needed, parameter to thr_lock
sql/lock.cc:
Fixed mysql_lock_tables() to retry with new sql_lock if lock fails. This was needed as table may be closed and reopened between retry's and then the old sql_lock will point to stale data.
sql/mysql_priv.h:
Updated prototype
sql/sql_base.cc:
Ensure that all tables are closed if opening of system table failes; This fixes the assert in THD::restore_backup_open_tables_state
sql/sql_handler.cc:
Updated variable type
Problem was that now we can merge derived table (subquery in the FROM clause).
Fix: in case of detected conflict and presence of derived table "over" the table which cased the conflict - try materialization strategy.
Fixed a typo in the comment.
Fixing test cases which were previouslyno throwing due
disable warnings macro.
sql/sql_base.cc:
Change in indentation and fixing a typo in the comment.
Problem: Statements that write to tables with auto_increment columns
based on the selection from another table, may lead to master
and slave going out of sync, as the order in which the rows
are retrieved from the table may differ on master and slave.
Solution: We mark writing to a table with auto_increment table
based on the rows selected from another table as unsafe. This
will cause the execution of such statements to throw a warning
and forces the statement to be logged in ROW if the logging
format is mixed.
Changes:
1. All the statements that writes to a table with auto_increment
column(s) based on the rows fetched from another table, will now
be unsafe.
2. CREATE TABLE with SELECT will now be unsafe.
sql/share/errmsg-utf8.txt:
Added new warning messages.
sql/sql_base.cc:
-Created function to check statements that write to
tables with auto_increment column and has select.
-Marked all the statements that write to a table
with auto_increment column based on rows fetched
from other table(s) as unsafe.
sql/sql_table.cc:
mark CREATE TABLE[with auto_increment column] as unsafe.
Problem: Statements that write to tables with auto_increment columns
based on the selection from another table, may lead to master
and slave going out of sync, as the order in which the rows
are retrived from the table may differ on master and slave.
Solution: We mark writing to a table with auto_increment table
as unsafe. This will cause the execution of such statements to
throw a warning and forces the statement to be logged in ROW if
the logging format is mixed.
Changes:
1. All the statements that writes to a table with auto_increment
column(s) based on the rows fetched from another table, will now
be unsafe.
2. CREATE TABLE with SELECT will now be unsafe.
sql/share/errmsg-utf8.txt:
Added new Warning messages
sql/sql_base.cc:
created a new function that checks for select + write on a autoinc table
made all such statements to be unsafe.
sql/sql_parse.cc:
made create autoincremnet tabble + select unsafe
A table expression with a natural join or a USING clause is transformed
into an equivalent expression with equi-join ON conditions. If a reference
to a virtual column happened to occur only in these generated equi-join
conditions then it was not erroneously marked in the TABLE::vcol_set bitmap.
This could lead to wrong results for queries containing natural join
expressions or USING clauses.
client/mysqltest.cc:
Free mutex after usage (fixes valgrind warnings in embedded server)
mysql-test/include/gis_keys.inc:
Fixed failure in innodb.gis_test
mysql-test/r/gis.result:
Updated result
mysql-test/suite/innodb/r/innodb_gis.result:
Updated results
mysql-test/suite/innodb/t/innodb_bug38231.test:
Added handling of timeouts (happend on some servers in buildbot)
mysql-test/suite/innodb_plugin/r/innodb_gis.result:
Updated results
mysql-test/suite/innodb_plugin/t/innodb.test:
Use error names instead of numbers
mysql-test/suite/innodb_plugin/t/innodb_misc1.test:
This test requires utf8
mysql-test/suite/innodb_plugin/t/innodb_mysql.test:
This test requires Xtradb
sql/sql_base.cc:
Don't print table names for placeholders.
sql/sql_show.cc:
Temporary fix:
Save and restore db and table_name in mysqld_show_create (to get rid of valgrind warning)
A better solution that needs to be investgated is to not change these fields in mysql_derived_prepare()
sql/sql_view.cc:
Fixed valgrind warning
storage/xtradb/handler/ha_innodb.cc:
Don't access THD directly
cmd-line-utils/libedit/map.c:
Fixed compiler warning
cmd-line-utils/libedit/terminal.c:
Fixed compiler warning
cmd-line-utils/libedit/tty.c:
Fixed compiler warning
sql/sql_base.cc:
Fixed memory leak found by valgrind
storage/maria/compat_aliases.cc:
Ensure that recover_alias is also a set
storage/maria/ma_bitmap.c:
Proper fix for compiler warning
support-files/compiler_warnings.supp:
Fixed compiler warning
sql/sql_base.cc:
fix a memory leak
storage/xtradb/handler/ha_innodb.cc:
fix for a visual studio
storage/xtradb/row/row0ins.c:
valgrind complains about uninitialized variable.
incorrect errors in the innodb.test too
A non-first execution of a prepared statement missed a call of the
TABLE_LIST::process_index_hints() method in the code of the function
setup_tables().
At some scenarios this could lead to the choice of a quite inefficient
execution plan for the base query of the prepared statement.
In MariaDB, when running in ONLY_FULL_GROUP_BY mode,
the server produced in incorrect error message that there
is an aggregate function without GROUP BY, for artificially
created MIN/MAX functions during subquery MIN/MAX optimization.
The fix introduces a way to distinguish between artifially
created MIN/MAX functions as a result of a rewrite, and normal
ones present in the query. The test for ONLY_FULL_GROUP_BY violation
now tests in addition if a MIN/MAX function was part of a MIN/MAX
subquery rewrite.
In order to be able to distinguish these MIN/MAX functions, the
patch introduces an additional flag in Item_in_subselect::in_strategy -
SUBS_STRATEGY_CHOSEN. This flag is set when the optimizer makes its
final choice of a subuqery strategy. In order to make the choice
consistent, access to Item_in_subselect::in_strategy is provided
via new class methods.
******
Fix MySQL BUG#12329653
In MariaDB, when running in ONLY_FULL_GROUP_BY mode,
the server produced in incorrect error message that there
is an aggregate function without GROUP BY, for artificially
created MIN/MAX functions during subquery MIN/MAX optimization.
The fix introduces a way to distinguish between artifially
created MIN/MAX functions as a result of a rewrite, and normal
ones present in the query. The test for ONLY_FULL_GROUP_BY violation
now tests in addition if a MIN/MAX function was part of a MIN/MAX
subquery rewrite.
In order to be able to distinguish these MIN/MAX functions, the
patch introduces an additional flag in Item_in_subselect::in_strategy -
SUBS_STRATEGY_CHOSEN. This flag is set when the optimizer makes its
final choice of a subuqery strategy. In order to make the choice
consistent, access to Item_in_subselect::in_strategy is provided
via new class methods.
NEW_FRM_MEM WITHOUT NEEDING TO".
During the process of opening tables for a statement, we allocated
memory which was used only during view loading even in cases when the
statement didn't use any views. Such an unnecessary allocation (and
corresponding freeing) might have caused significant performance
overhead in some workloads. For example, it caused up to 15% slowdown
in a simple stored routine calculating Fibonacci's numbers.
This memory was pre-allocated as part of "new_frm_mem" MEM_ROOT
initialization at the beginning of open_tables().
This patch addresses this issue by turning off memory pre-allocation
during initialization for this MEM_ROOT. Now, memory on this root
will be allocated only at the point when the first .FRM for a view is
opened.
The patch doesn't contain a test case since it is hard to test the
performance improvements or the absence of memory allocation in our
test framework.
sql/sql_insert.cc:
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
******
CREATE ... IF NOT EXISTS may do nothing, but
it is still not a failure. don't forget to my_ok it.
sql/sql_table.cc:
small cleanup
******
small cleanup
- If USER is given, all threads for that user is signaled
- If SOFT is used then the KILL will not be sent to the handler. This can be used to not interrupt critical things in the handler like 'REPAIR'.
Internally added more kill signals. This gives us more information of why a query/connection was killed.
- KILL_SERVER is used when server is going down. In this case the users gets ER_SHUTDOWN as the reason connection was killed.
- Changed signals to number in correct order, which makes it easier to test how the signal should affect the code.
- New error message ER_CONNECTION_KILLED if connection was killed by 'KILL CONNECTION'. Before we got error ER_SHUTDOWN.
Changed names of not used parameters KILL_QUERY & KILL_CONNCTION to mysql_kill() to not conflict with defines in the server
include/mysql.h.pp:
Updated file
include/mysql_com.h:
Changed names of not used parameters KILL_QUERY & KILL_CONNCTION to mysql_kill() to not conflict with defines in the server
mysql-test/r/kill.result:
Added test of KILL USER
mysql-test/suite/rpl/r/rpl_stm_000001.result:
Updated error code
mysql-test/suite/rpl/t/rpl_stm_000001.test:
Updated error codes
mysql-test/t/flush_read_lock_kill.test:
Updated error codes
mysql-test/t/kill.test:
Added test of KILL USER
plugin/handler_socket/handlersocket/database.cpp:
Removed THD:: from KILL
sql/debug_sync.cc:
Removed THD:: from KILL
sql/event_scheduler.cc:
Removed THD:: from KILL
sql/filesort.cc:
Removed THD:: from KILL
sql/ha_ndbcluster_binlog.cc:
Removed THD:: from KILL
sql/handler.cc:
Removed THD:: from KILL
Simplify code.
sql/lex.h:
Added new keywords HARD | SOFT
sql/log.cc:
Removed THD:: from KILL
Added testing of new error ER_CONNECTION_KILLED
sql/log_event.cc:
Removed THD:: from KILL
Added testing of new error ER_CONNECTION_KILLED
sql/mysql_priv.h:
Added new prototypes
sql/mysqld.cc:
Removed THD:: from KILL
Use KILL_SERVER_HARD signal on shutdown.
sql/scheduler.cc:
Removed THD:: from KILL
Simplify test if connection should be killed
sql/share/errmsg.txt:
New error message ER_CONNECTION_KILLED
sql/slave.cc:
Removed THD:: from KILL
sql/sp_head.cc:
Removed THD:: from KILL
sql/sql_base.cc:
Removed THD:: from KILL
sql/sql_cache.cc:
Removed THD:: from KILL
sql/sql_class.cc:
Removed THD:: from KILL
Added killed_errno()
Only signal kill to storage engine if HARD bit is set.
sql/sql_class.h:
Move KILL options out from THD to make them easier to use in sql_yacc.yy
sql/sql_connect.cc:
Removed THD:: from KILL
sql/sql_delete.cc:
Removed THD:: from KILL
sql/sql_error.cc:
Removed THD:: from KILL
sql/sql_insert.cc:
Removed THD:: from KILL
Simplifed testing if thread is killed.
sql/sql_lex.h:
Added kill options to st_lex
sql/sql_load.cc:
Removed THD:: from KILL
sql/sql_parse.cc:
Added kill options to st_lex
Simplifed and optimzed testing of thd->killed at end of query
Added support for KILL USER
Extended sql_kill() to allow use of more kill signals.
sql/sql_repl.cc:
Removed THD:: from KILL
sql/sql_show.cc:
Removed THD:: from KILL
Simplied testing if query/connection was killed
sql/sql_table.cc:
Removed THD:: from KILL
sql/sql_update.cc:
Removed THD:: from KILL
sql/sql_yacc.yy:
Added support for new KILL syntax: KILL [HARD|SOFT] [CONNECTION|QUERY] [ID | USER user_name]
storage/archive/ha_archive.cc:
Simplify compilation
storage/maria/ha_maria.cc:
Removed THD:: from KILL
- The problem was that Item_direct_view_ref and its embedded Item_field were getting incorrect
value of item->used_tables() after fix_fields() in the second and subsequent EXECUTE.
- Made relevant fixes in Item_field::fix_fields() and find_field_in_tables(), so that the
Item_field gets the correct attributes.
Give more information when finding an error in a MyISAM table.
When killing system thread, use KILL_SYSTEM_THREAD instead of KILL_CONNECTION to make it easier to ignore the signal in sensitive context (like auto-repair)
Added new kill level: KILL_SERVER that will in the future to be used to signal killed by shutdown.
Add more warnings about killed connections when warning level > 3
include/myisamchk.h:
Added counting of printed info/notes
mysys/mf_iocache.c:
Remove duplicate assignment
sql/handler.cc:
Added test of KILL_SERVER
sql/log.cc:
Ignore new 'kill' error ER_NEW_ABORTING_CONNECTION when requesting query error code.
sql/mysqld.cc:
Add more warnings for killed connections when warning level > 3
sql/scheduler.cc:
Added checks for new kill signals
sql/slave.cc:
Ignore new kill signal ER_NEW_ABORTING_CONNECTION
sql/sp_head.cc:
Fixed assignment to bool
Added testing of new kill signals
sql/sql_base.cc:
Use KILL_SYSTEM_THREAD to auto-kill system threads
sql/sql_class.cc:
Add more warnings for killed connections when warning level > 3
thd_killed() now ignores KILL_BAD_DATA and THD::KILL_SYSTEM_THREAD as these should not abort sensitive operations.
sql/sql_class.h:
Added KILL_SYSTEM_THREAD and KILL_SERVER
sql/sql_connect.cc:
Added handling of KILL_SERVER
sql/sql_insert.cc:
Use KILL_SYSTEM_THREAD to auto-kill system threads
Added handling of KILL_SERVER
sql/sql_parse.cc:
Add more warnings for killed connections when warning level > 3
Added checking that thd->abort_on_warning is reset at end of query.
sql/sql_show.cc:
Update condition for when a query is 'killed'
storage/myisam/ha_myisam.cc:
Added counting of info/notes printed
storage/myisam/mi_check.c:
Always print an an error if we find data errors when checking/repairing a MyISAM table.
When a repair was killed, don't retry repair.
Added assert if sort_get_next_record() returned an error without an error message.
Removed nonsence check "if (sort_param->read_cache.error < 0)" in repair.
storage/myisam/myisamchk.c:
Added counting of notes printed
storage/pbxt/src/thread_xt.cc:
Better error message.
The idea is that my_global_flags is ored to the MyFlags parameter for the above functions if the MY_WME flag is not set.
As the my_global_flags has ME_JUST_INFO (mark error as 'note') and possible ME_NOREFRESH (write error to log) this will force mysqld to log the not critical error to the log as a note.
include/my_sys.h:
Moved MY_SYNC_DIR to ensure it never clashes with ME_JUST_INFO
Added my_global_flags
mysql-test/Makefile.am:
Removed not used bugs directory
mysys/my_init.c:
Added my_global_flags, a variable that is ored to MyFlags in a those mysys functions we want extra logging.
mysys/my_malloc.c:
Added support for my_global_flags
mysys/my_open.c:
Added support for my_global_flags
mysys/my_pread.c:
Added support for my_global_flags
mysys/my_read.c:
Added support for my_global_flags
mysys/my_static.c:
Added my_global_flags
mysys/my_write.c:
Added support for my_global_flags
sql/mysqld.cc:
Set my_global_flags for warning levels 10 & 11
sql/sql_base.cc:
Don't increment unhandled errors for notes or warnings.
- The problem was that the code that made the check whether the subquery is an AND-part of the WHERE
clause didn't work correctly for nested subqueries. In particular, grand-child subquery in HAVING was
treated as if it was in the WHERE, which eventually caused an assert when replace_where_subcondition
looked for the subquery predicate in the WHERE and couldn't find it there.
- The fix: Removed implementation of "thd_marker approach". thd->thd_marker was used to determine the
location of subquery predicate: setup_conds() would set accordingly it when making the
{where|on_expr}->fix_fields(...)
call so that AND-parts of the WHERE/ON clauses can determine they are the AND-parts.
Item_cond_or::fix_fields(), Item_func::fix_fields(), Item_subselect::fix_fields (this one was missed),
and all other items-that-contain-items had to reset thd->thd_marker before calling fix_fields() for
their children items, so that the children can see they are not AND-parts of WHERE/ON.
- The "thd_marker approach" required that a lot of code in different locations maintains correct value of
thd->thd_marker, so it was replaced with:
- The new approach with mark_as_condition_AND_part does not keep context in thd->thd_marker. Instead,
setup_conds() now calls
{where|on_expr}->mark_as_condition_AND_part()
and implementations of that function make sure that:
- parts of AND-expressions get the mark_as_condition_AND_part() call
- Item_in_subselect objects record that they are AND-parts of WHERE/ON
