This is a performance bug, affecting in particular the bison generated code
for the parser.
Prior to this fix, the grammar used a long chain of reduces to parse an
expression, like:
bit_expr -> bit_term
bit_term -> bit_factor
bit_factor -> value_expr
value_expr -> term
term -> factor
etc
This chain of reduces cause the internal state automaton in the generated
parser to execute more state transitions and more reduces, so that the
generated MySQLParse() function would spend a lot of time looping to execute
all the grammar reductions.
With this patch, the grammar has been reorganized so that rules are more
"flat", limiting the depth of reduces needed to parse <expr>.
Tests have been written to enforce that relative priorities and properties
of operators have not changed while changing the grammar.
See the bug report for performance data.
Currently the Last_query_cost session status variable shows
only the cost of a single flat subselect. For complex queries
(with subselects or unions etc) Last_query_cost is not valid
as it was showing the cost for the last optimized subselect.
Fixed by reseting to zero Last_query_cost when the complete
cost of the query cannot be determined.
Last_query_cost will be non-zero only for single flat queries.
The optimization that uses a unique index to remove GROUP BY, did not
ensure that the index was actually used, thus violating the ORDER BY
that is impled by GROUP BY.
Fixed by replacing GROUP BY with ORDER BY if the GROUP BY clause contains
a unique index. In case GROUP BY ... ORDER BY null is used, GROUP BY is
simply removed.
If, after the tables are locked, one of the conditions to read from a
HANDLER table is not met, the handler code wrongly jumps to a error path
that won't unlock the tables.
The user-visible effect is that after a error in a handler read command,
all subsequent handler operations on the same table will hang.
The fix is simply to correct the code to jump to the (same) error path that
unlocks the tables.
The problem from a user's perspective: user creates table A, and then tries
to CREATE TABLE a SELECT from A - and this causes a deadlock error, a hang,
or fails with a debug assert, but only if the storage engine is InnoDB.
The origin of the problem: InnoDB uses case-insensitive collation
(system_charset_info) when looking up the internal table share, thus returning
the same share for 'a' and 'A'.
Cause of the user-visible behavior: since the same share is returned to SQL
locking subsystem, it assumes that the same table is first locked (within the
same session) for WRITE, and then for READ, and returns a deadlock error.
However, the code is wrong in not properly cleaning up upon an error, leaving
external locks in place, which leads to assertion failures and hangs.
Fix that has been implemented: the SQL layer should properly propagate the
deadlock error, cleaning up and freeing all resources.
Further work towards a more complete solution: InnoDB should not use case
insensitive collation for table share hash if table names on disk honor the case.
This is a performance bug, related to the parsing or 'OR' and 'AND' boolean
expressions.
Let N be the number of expressions involved in a OR (respectively AND).
When N=1
For example, "select 1" involve only 1 term: there is no OR operator.
In 4.0 and 4.1, parsing expressions not involving OR had no overhead.
In 5.0, parsing adds some overhead, with Select->expr_list.
With this patch, the overhead introduced in 5.0 has been removed,
so that performances for N=1 should be identical to the 4.0 performances,
which are optimal (there is no code executed at all)
The overhead in 5.0 was in fact affecting significantly some operations.
For example, loading 1 Million rows into a table with INSERTs,
for a table that has 100 columns, leads to parsing 100 Millions of
expressions, which means that the overhead related to Select->expr_list
is executed 100 Million times ...
Considering that N=1 is by far the most probable expression,
this case should be optimal.
When N=2
For example, "select a OR b" involves 2 terms in the OR operator.
In 4.0 and 4.1, parsing expressions involving 2 terms created 1 Item_cond_or
node, which is the expected result.
In 5.0, parsing these expression also produced 1 node, but with some extra
overhead related to Select->expr_list : creating 1 list in Select->expr_list
and another in Item_cond::list is inefficient.
With this patch, the overhead introduced in 5.0 has been removed
so that performances for N=2 should be identical to the 4.0 performances.
Note that the memory allocation uses the new (thd->mem_root) syntax
directly.
The cost of "is_cond_or" is estimated to be neglectable: the real problem
of the performance degradation comes from unneeded memory allocations.
When N>=3
For example, "select a OR b OR c ...", which involves 3 or more terms.
In 4.0 and 4.1, the parser had no significant cost overhead, but produced
an Item tree which is difficult to evaluate / optimize during runtime.
In 5.0, the parser produces a better Item tree, using the Item_cond
constructor that accepts a list of children directly, but at an extra cost
related to Select->expr_list.
With this patch, the code is implemented to take the best of the two
implementations:
- there is no overhead with Select->expr_list
- the Item tree generated is optimized and flattened.
This is achieved by adding children nodes into the Item tree directly,
with Item_cond::add(), which avoids the need for temporary lists and memory
allocation
Note that this patch also provide an extra optimization, that the previous
code in 5.0 did not provide: expressions are flattened in the Item tree,
based on what the expression already parsed is, and not based on the order
in which rules are reduced.
For example : "(a OR b) OR c", "a OR (b OR c)" would both be represented
with 2 Item_cond_or nodes before this patch, and with 1 node only with this
patch. The logic used is based on the mathematical properties of the OR
operator (it's associative), and produces a simpler tree.
Killing a SELECT query with KILL QUERY or KILL CONNECTION
causes a server crash if the query cache is enabled.
Normal evaluation of a query may be interrupted by the
KILL QUERY/CONNECTION statement, in this case the mysql_execute_command
function returns TRUE, and the thd->killed flag has true value.
In this case the result of the query may
be cached incompletely (omitting call to query_cache_insert inside
the net_real_write function), and next call to query_cache_end_of_result
may lead to server crash.
Thus, the query_cache_end_of_result function has been modified to abort
query cache in the case of killed thread.
Although the query cache doesn't support retrieval of statements containing
column level access control, it was still possible to cache such statements
thus wasting memory.
This patch extends the access control check on the target tables to avoid
caching a statement with column level restrictions.
Views are excepted and can be cached but only retrieved by super user account.
Although the query cache doesn't support retrieval of statements containing
column level access control, it was still possible to cache such statements
thus wasting memory.
This patch extends the access control check on the target tables to avoid
caching a statement with column level restrictions.
HEAP tables can't index BIT fields. Due to this when grouping by such fields is
needed they are converted to a fields of the LONG type when temporary table
is being created. But a side effect of this is that a wrong type of BIT
fields is returned to a client.
Now the JOIN::prepare and the create_distinct_group functions are create
additional hidden copy of BIT fields to preserve original fields untouched.
New hidden fields are used for grouping instead.
This is a follow up for the patch for Bug#26162 "Trigger DML ignores low_priority_updates setting", where the stored procedure ignores the session setting of low_priority_updates.
For every table open operation with default write (TL_WRITE_DEFAULT) lock_type, downgrade the lock type to the session setting of low_priority_updates.
Apply innodb-5.0-ss1696 snapshot
Fixes:
- Bug#20090: InnoDB: Error: trying to declare trx to enter InnoDB
- Bug#23710: crash_commit_before fails if innodb_file_per_table=1
At InnoDB startup consider the case where log scan went beyond
checkpoint_lsn as a crash and initiate crash recovery code path.
- Bug#28781: InnoDB increments auto-increment value incorrectly with ON DUPLICATE KEY UPDATE
We need to do some special AUTOINC handling for the following case:
INSERT INTO t (c1,c2) VALUES(x,y) ON DUPLICATE KEY UPDATE ...
We need to use the AUTOINC counter that was actually used by
MySQL in the UPDATE statement, which can be different from the
value used in the INSERT statement.
- Bug#29097: fsp_get_available_space_in_free_extents() is capped at 4TB
Fix by typecasting the variables before multiplying them, so that the
result of the multiplication is of type "unsigned long long".
- Bug#29155: Innodb "Parallel recovery" is not prevented
Fix by enabling file locking on FreeBSD. It has been disabled because
InnoDB has refused to start on FreeBSD & LinuxThreads, but now it
starts just fine.
The bug caused memory corruption for some queries with top OR level
in the WHERE condition if they contained equality predicates and
other sargable predicates in disjunctive parts of the condition.
The corruption happened because the upper bound of the memory
allocated for KEY_FIELD and SARGABLE_PARAM internal structures
containing info about potential lookup keys was calculated incorrectly
in some cases. In particular it was calculated incorrectly when the
WHERE condition was an OR formula with disjuncts being AND formulas
including equalities and other sargable predicates.
mysql_ha_open calls mysql_ha_close on the error path (unsupported) to close the (opened) table before inserting it into the tables hash list handler_tables_hash) but mysql_ha_close only closes tables which are on the hash list, causing the table to be left open and locked.
This change moves the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close (mysql_ha_close).
ORDER BY is used
The range analysis module did not correctly signal to the
handler that a range represents a ref (EQ_RANGE flag). This causes
non-range queries like
SELECT ... FROM ... WHERE keypart_1=const, ..., keypart_n=const
ORDER BY ... FOR UPDATE
to wait for a lock unneccesarily if another running transaction uses
SELECT ... FOR UPDATE on the same table.
Fixed by setting EQ_RANGE for all range accesses that represent
an equality predicate.
under terms of bug#28875 for better performance.
The change appeared to require more changes in item_cmpfunc.cc,
which is dangerous in 5.0.
Conversion between a latin1 column and an ascii string constant
stopped to work.
Two character mappings were way off (backtick and tilde were "E"
and "Y"!), and three others were slightly rotated. The first
would cause collisions, and the latter was probably benign.
Now, assign the character mappings exactly to their normal values.
MySQL replicates the time zone only when operations that involve
it are performed. This is controlled by a flag. But this flag
is set only on successful operation.
The flag must be set also when there is an error that involves
a timezone (so the master would replicate the error to the slaves).
Fixed by moving the setting of the flag before the operation
(so it apples to errors as well).
VIEW".
mysql_list_fields() C API function would incorrectly set MYSQL_FIELD::decimals
member for some view columns.
The problem was in an incomplete implementation of
Item_ident_for_show::make_field(), which is responsible for view
columns metadata.
This bug manifested itself for queries with grouping by columns of
the BIT type. It led to wrong comparisons of bit-field values and
wrong result sets.
Bit-field values never cannot be compared as binary values. Yet
the class Field_bit had an implementation of the cmp method that
compared bit-fields values as binary values.
Also the get_image and set_image methods of the base class Field
cannot be used for objects of the Field_bit class.
Now these methods are declared as virtual and specific implementations
of the methods are provided for the class Field_bit.
(Regression, caused by a patch for the bug 22646).
Problem: when result type of date_format() was changed from
binary string to character string, mixing date_format()
with a ascii column in CONCAT() stopped to work.
Fix:
- adding "repertoire" flag into DTCollation class,
to mark items which can return only pure ASCII strings.
- allow character set conversion from pure ASCII to other character sets.
The SELECT query with more than 31 nested dependent SELECT queries returned
wrong result.
New error message has been added: ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT.
It will be reported as: "Too high level of nesting for select".
When DROP TABLE detects that it has been killed
by another thread it must unlock the table names
it locked.
Fixed by backporting the 5.1 error handling code.
Aligned client library build and use with the Unix version when it
comes to what source to include directly in the builds, and what
libraries to link with (bug#30118).
Also reviewed, corrected and made more clear when static or dynamic
Thread Local Storage is to be used. Some code duplication was removed,
and some redundant library usage were removed, reducing the risk of
incorrect TLS usage.
Problem: thd->thread_specific_used flag is not set executing a statement
containig connection_id() function using PS protocol, that leads to
improper binlog event creation.
Fix: set the flag in the Item_func_connection_id::fix_fields().
restores from mysqlbinlog out
Problem: using "mysqlbinlog | mysql" for recoveries the connection_id()
result may differ from what was used when issuing the statement.
Fix: if there is a connection_id() in a statement, write to binlog
SET pseudo_thread_id= XXX; before it and use the value later on.
INSERT DELAYED on a replication slave was converted to regular INSERT,
whereas it should try concurrent INSERT first.
With this patch we try to convert delayed insert to concurrent insert on
a replication slave. If it is impossible for some reason, we fall back to
regular insert.
No test case for this fix. I do not see anything indicating this is
regression - we behave this way since Nov 2000.
- Don't call mysql_select() several times for the select that enumerates
a temporary table with the results of the UNION. Making this call for
every subquery execution caused O(#enumerated-rows-in-the-outer-query)
memory allocations.
- Instead, call join->reinit() and join->exec(), and
= disable constant table detection for such joins,
= provide special handling for table-less constant subqueries.
SELECT statement itself returns empty.
As a result of this bug 'SELECT AGGREGATE_FUNCTION(fld) ... GROUP BY'
can return one row instead of an empty result set.
When GROUP BY only has fields of constant tables
(with a single row), the optimizer deletes the group_list.
After that we lose the information about whether we had an
GROUP BY statement. Though it's important
as SELECT min(x) from empty_table; and
SELECT min(x) from empty_table GROUP BY y; have to return
different results - the first query should return one row,
second - an empty result set.
So here we add the 'group_optimized_away' flag to remember this case
when GROUP BY exists in the query and is removed
by the optimizer, and check this flag in end_send_group()
Bug #27417 thd->no_trans_update.stmt lost value inside of SF-exec-stack
Once had been set the flag might later got reset inside of a stored routine
execution stack.
The reason was in that there was no check if a new statement started at time
of resetting.
The artifact affects most of binlogable DML queries. Notice, that multi-update
is wrapped up within
bug@27716 fix, multi-delete bug@29136.
Fixed with saving parent's statement flag of whether the statement modified
non-transactional table, and unioning (merging) the value with that was gained
in mysql_execute_command.
Resettling thd->no_trans_update members into thd->transaction.`member`;
Asserting code;
Effectively the following properties are held.
1. At the end of a substatement thd->transaction.stmt.modified_non_trans_table
reflects the fact if such a table got modified by the substatement.
That also respects THD::really_abort_on_warnin() requirements.
2. Eventually thd->transaction.stmt.modified_non_trans_table will be computed as
the union of the values of all invoked sub-statements.
That fixes this bug#27417;
Computing of thd->transaction.all.modified_non_trans_table is refined to base to
the stmt's value for all the case including insert .. select statement which
before the patch had an extra issue bug@28960.
Minor issues are covered with mysql_load, mysql_delete, and binloggin of insert in
to temp_table select.
The supplied test verifies limitely, mostly asserts. The ultimate testing is defered
for bug@13270, bug@23333.
When innodb detects a deadlock it calls ha_rollback_trans() to rollback the
main transaction. But such action isn't allowed from inside of triggers and
functions. When it happen the 'Explicit or implicit commit' error is thrown
even if there is no commit/rollback statements in the trigger/function. This
leads to the user confusion.
Now the convert_error_code_to_mysql() function doesn't call the
ha_rollback_trans() function directly but rather calls the
mark_transaction_to_rollback function and returns an error.
The sp_rcontext::find_handler() now doesn't allow errors to be caught by the
trigger/function error handlers when the thd->is_fatal_sub_stmt_error flag
is set. Procedures are still allowed to catch such errors.
The sp_rcontext::find_handler function now accepts a THD handle as a parameter.
The transaction_rollback_request and the is_fatal_sub_stmt_error flags are
added to the THD class. The are initialized by the THD class constructor.
Now the ha_autocommit_or_rollback function rolls back main transaction
when not in a sub statement and the thd->transaction_rollback_request
is set.
The THD::restore_sub_statement_state function now resets the
thd->is_fatal_sub_stmt_error flag on exit from a sub-statement.
SP with local variables with non-ASCII names crashed the server.
The server replaces SP local variable names with NAME_CONST calls
when putting statements into the binary log. It used UTF8-encoded
item names as variable names for the replacement inside NAME_CONST
calls. However, statement string may be encoded by any
known character set by the SET NAMES statement.
The server used byte length of UTF8-encoded names to increment
the position in the query string that led to array index overrun.
information schema table.
The get_schema_views_record() function fills records in the view table of
the informations schema with data about given views. Among other info
the is_updatable flag is set. But the check whether the view is updatable or
not wasn't covering all cases thus sometimes providing wrong info.
This might led to a user confusion.
Now the get_schema_views_record function additionally calls to the
view->can_be_merge() function to find out whether the view can be updated or
not.
The subst_spvars function is used to create query string with SP variables
substituted with their values. This string is used later for the binary log
and for the query cache. The problem is that the
query_cache_send_result_to_client function requires some additional space
after the query to store database name and query cache flags. This
space wasn't reserved by the subst_spvars function which led to a memory
corruption and crash.
Now the subst_spvars function reserves additional space for the query cache.
When a table was explicitly locked with LOCK TABLES no associated
tables from any related trigger on the subject table were locked.
As a result of this the user could experience unexpected locking
behavior and statement failures similar to "failed: 1100: Table'xx'
was not locked with LOCK TABLES".
This patch fixes this problem by making sure triggers are
pre-loaded on any statement if the subject table was explicitly
locked with LOCK TABLES.
Item_func_user doesn't calculate anything in it's val_str() method,
just returns saved str_value.
Though Item::save_in_field method can destroy str_value, relying on
val_str() return. As a result we get the garbage stored in field.
We cannot use Item::save_in_field implementation for Item_func_user,
reimplement it in simpler way.
between perm and temp tables. Review fixes.
The original bug report complains that if we locked a temporary table
with LOCK TABLES statement, we would not leave LOCK TABLES mode
when this temporary table is dropped.
Additionally, the bug was escalated when it was discovered than
when a temporary transactional table that was previously
locked with LOCK TABLES statement was dropped, futher actions with
this table, such as UNLOCK TABLES, would lead to a crash.
The problem originates from incomplete support of transactional temporary
tables. When we added calls to handler::store_lock()/handler::external_lock()
to operations that work with such tables, we only covered the normal
server code flow and did not cover LOCK TABLES mode.
In LOCK TABLES mode, ::external_lock(LOCK) would sometimes be called without
matching ::external_lock(UNLOCK), e.g. when a transactional temporary table
was dropped. Additionally, this table would be left in the list of LOCKed
TABLES.
The patch aims to address this inadequacy. Now, whenever an instance
of 'handler' is destroyed, we assert that it was priorly
external_lock(UNLOCK)-ed. All the places that violate this assert
were fixed.
This patch introduces no changes in behavior -- the discrepancy in
behavior will be fixed when we start calling ::store_lock()/::external_lock()
for all tables, regardless whether they are transactional or not,
temporary or not.
- make ha_berkeley::cmp_ref() take into account that auto-generated PKs
are stored in LSB-first order.
- Remove the temporary code that made the bugfix work for innodb only
to 150 or 107 characters for those messages which are generated
by the embedded server during release builds.
This fixes bug#16635:
Error messages wrong: absolute path names, "%s" format code
See the bug report or the changelog for "sql/share/english/errmsg.txt"
for instructions how to do that with other languages,
even at the customer site, and for the restrictions to keep.
If ::exteral_lock hadn't been called, we'd have no NDB object,
so need to check/get one here.
It looks like sql_show.cc is the only place that does this....
or at least the other places will be well hidden.
Index: ndb-work/sql/ha_ndbcluster.cc
===================================================================
the master but on the slave
MySQL can decide to "downgrade" a INSERT DELAYED statement
to normal insert in certain situations.
One such situation is when the slave is replaying a
replication feed.
However INSERT DELAYED is logged even if there're no updates
whereas the NORMAL INSERT is not logged in such cases.
Fixed by always logging a "downgraded" INSERT DELAYED: even
if there were no updates.
"Federated Denial of Service"
Federated storage engine used to attempt to open connections within
the ::create() and ::open() methods which are invoked while LOCK_open
mutex is being held by mysqld. As a result, no other client sessions
can open tables while Federated is attempting to open a connection.
Long DNS lookup times would stall mysqld's operation and a rogue
connection string which connects to a remote server which simply
stalls during handshake can stall mysqld for a much longer period of
time.
This patch moves the opening of the connection much later, when the
federated actually issues queries, by which time the LOCK_open mutex is
no longer being held.
When the SQL_BIG_RESULT flag is specified SELECT should store items from the
select list in the filesort data and use them when sending to a client.
The get_addon_fields function is responsible for creating necessary structures
for that. But this function was allowed to do so only for SELECT and
INSERT .. SELECT queries. This makes the SQL_BIG_RESULT useless for
the CREATE .. SELECT queries.
Now the get_addon_fields allows storing select list items in the filesort
data for the CREATE .. SELECT queries.
"getGeneratedKeys() does not work with FEDERATED table"
mysql_insert() expected the storage engine to update the row data
during the write_row() operation with the value of the new auto-
increment field. The field must be updated when only one row has
been inserted as mysql_insert() would ignore the thd->last_insert.
This patch implements HA_STATUS_AUTO support in ha_federated::info()
and ensures that ha_federated::write_row() does update the row's
auto-increment value.
The test case was written in C as the protocol's 'id' value is
accessible through libmysqlclient and not via SQL statements.
mysql-test-run.pl was extended to enable running the test binary.
