This is a performance bug, affecting in particular the bison generated code
for the parser.
Prior to this fix, the grammar used a long chain of reduces to parse an
expression, like:
bit_expr -> bit_term
bit_term -> bit_factor
bit_factor -> value_expr
value_expr -> term
term -> factor
etc
This chain of reduces cause the internal state automaton in the generated
parser to execute more state transitions and more reduces, so that the
generated MySQLParse() function would spend a lot of time looping to execute
all the grammar reductions.
With this patch, the grammar has been reorganized so that rules are more
"flat", limiting the depth of reduces needed to parse <expr>.
Tests have been written to enforce that relative priorities and properties
of operators have not changed while changing the grammar.
See the bug report for performance data.
mysql-test/r/parser_precedence.result:
Improved test coverage for operator precedence
mysql-test/t/parser_precedence.test:
Improved test coverage for operator precedence
sql/sql_yacc.yy:
Simplified the grammar to improve performances
The optimization that uses a unique index to remove GROUP BY did not
ensure that the index was actually used, thus violating the ORDER BY
that is implied by GROUP BY.
Fixed by replacing GROUP BY with ORDER BY if the GROUP BY clause contains
a unique index over non-nullable field(s). In case GROUP BY ... ORDER BY
null is used, GROUP BY is simply removed.
mysql-test/include/mix1.inc:
Bug#30596: Test case for InnoDB
Here, as opposed to for MyISAM, row lookup is done using index
whenever the index covers the group list.
mysql-test/r/distinct.result:
Bug#30596: Changed test case.
Prior to Bug#16458, These queries use temp table and filesort. The
bug was that they used a temp table. However, that patch removed
filesort also, in which case we can no longer gurantee correct ordering.
mysql-test/r/group_by.result:
Bug#30596: Correct result
The test case for IGNORE INDEX FOR GROUP BY gets degraded performance
(unneccesary filesort). This is due to Bug#30665, which will be fixed separately.
mysql-test/r/innodb_mysql.result:
Bug#30596: Test result
mysql-test/t/group_by.test:
Bug#30596: Test case
sql/sql_select.cc:
Bug#30596: The fix:
- replace GROUP BY with ORDER BY unless ORDER BY [NULL|<constant>]
- make sure to use the keys for GROUP BY in this ORDER BY.
Currently the Last_query_cost session status variable shows
only the cost of a single flat subselect. For complex queries
(with subselects or unions etc) Last_query_cost is not valid
as it was showing the cost for the last optimized subselect.
Fixed by reseting to zero Last_query_cost when the complete
cost of the query cannot be determined.
Last_query_cost will be non-zero only for single flat queries.
mysql-test/r/status.result:
Bug #30377: test case
mysql-test/t/status.test:
Bug #30377: test case
sql/sql_lex.h:
Bug #30377: helper function
sql/sql_select.cc:
Bug #30377: don't assign cost if not on single level statement
into pilot.(none):/data/msvensson/mysql/mysql-5.1-new-maint
client/mysqltest.c:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/t/loaddata.test:
Auto merged
- Merge sslaccept and sslconnect.
- Atomically "reset" vio to VIO_TYPE_SSL when the SSL connection has
succeeded, this avoids having to revert anything and thus protects
against "close_active_vio" in the middle.
- Add some variance to the testcase
mysql-test/t/rpl_ssl.test:
Add some variance by running two selects before stopping the slave
Check that number of records in t1 are equal on master and slave
vio/viossl.c:
Rewrite sslconnect and sslaccept to automically "reset" the vio
to VIO_TYPE_SSL. Also use the fd from 'SSL_get_fd' to avoid
setting vio->sd to -1, that previously occured when "close_active_vio"
was called during connect/accept.
Merge the two function since they were exactly the same except for one line.
Update the DBUG printouts to be generic(i.e use peer instead of client/server).
into whalegate.ndb.mysql.com:/home/tomas/mysql-5.1-new-ndb-merge
sql/field.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/rpl_record.cc:
Auto merged
sql/rpl_utility.cc:
Auto merged
sql/rpl_utility.h:
Auto merged
into ramayana.hindu.god:/home/tsmith/m/bk/maint/51
configure.in:
Auto merged
mysql-test/r/innodb.result:
Auto merged
mysql-test/r/partition_pruning.result:
Auto merged
mysql-test/r/select.result:
Auto merged
mysql-test/suite/rpl/include/rpl_mixed_ddl.inc:
Auto merged
mysql-test/suite/rpl/include/rpl_mixed_dml.inc:
Auto merged
mysql-test/suite/rpl/r/rpl_timezone.result:
Auto merged
mysql-test/suite/rpl/t/rpl_relayspace.test:
Auto merged
mysql-test/suite/rpl/t/rpl_timezone.test:
Auto merged
mysql-test/t/select.test:
Auto merged
netware/BUILD/compile-netware-END:
Auto merged
netware/Makefile.am:
Auto merged
sql/field.cc:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/item.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_cache.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_view.cc:
Auto merged
sql/time.cc:
Auto merged
storage/innobase/handler/ha_innodb.cc:
Auto merged
storage/innobase/log/log0recv.c:
Auto merged
storage/innobase/srv/srv0srv.c:
Auto merged
storage/innobase/trx/trx0trx.c:
Auto merged
mysql-test/suite/rpl/r/rpl_innodb_mixed_dml.result:
manual merge (use remote)
mysql-test/suite/rpl/t/disabled.def:
Manual merge - both rpl_innodb_mixed_ddl and rpl_innodb_mixed_dml
are fixed, and should be run as part of the rpl suite.
into ramayana.hindu.god:/home/tsmith/m/bk/maint/51
client/mysql_upgrade.c:
Auto merged
client/mysqltest.c:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
sql/sql_show.cc:
Auto merged
Initialize thd->variables.pseudo_thread_id when a new embedded
thd is created.
libmysqld/lib_sql.cc:
Add comment regarding duplication of code in create_embedded_thd()
vs. create_new_thread() and prepare_new_connection_state(). This
was a cause for not properly initializing the pseudo_thread_id variable.
mysql-test/r/func_misc.result:
Add test case to ensure connection_id() returns a sane value
mysql-test/t/func_misc.test:
Add test case to ensure connection_id() returns a sane value
sql/mysqld.cc:
Add comment warning of the duplication of code between create_new_thread()
and create_embedded_thd()
sql/sql_connect.cc:
Add comment warning of the duplication of code between
prepare_new_connection_state() and create_embedded_thd()
The optimization that uses a unique index to remove GROUP BY, did not
ensure that the index was actually used, thus violating the ORDER BY
that is impled by GROUP BY.
Fixed by replacing GROUP BY with ORDER BY if the GROUP BY clause contains
a unique index. In case GROUP BY ... ORDER BY null is used, GROUP BY is
simply removed.
BitKeeper/etc/ignore:
Added support-files/mysqld_multi.server tests/bug25714 cscope.in.out cscope.out cscope.po.out to the ignore list
mysql-test/r/distinct.result:
Bug#30596: Changed test case.
Prior to Bug#16458, These queries use temp table and filesort. The
bug was that they used a temp table. However, that patch removed
filesort also, in which case we can no longer gurantee correct ordering.
mysql-test/r/group_by.result:
Bug#30596: Correct result
mysql-test/r/innodb_mysql.result:
Bug#30596: Test case for innodb. Here, as opposed to for MyISAM, row
lookup is done using index whenever the index covers the group list.
mysql-test/t/group_by.test:
Bug#30596: Test case
mysql-test/t/innodb_mysql.test:
Bug#30596: Test case
sql/sql_select.cc:
Bug#30596: The fix, replacing GROUP BY with ORDER BY unless
ORDER BY [NULL|<constant>]
If, after the tables are locked, one of the conditions to read from a
HANDLER table is not met, the handler code wrongly jumps to a error path
that won't unlock the tables.
The user-visible effect is that after a error in a handler read command,
all subsequent handler operations on the same table will hang.
The fix is simply to correct the code to jump to the (same) error path that
unlocks the tables.
mysql-test/r/handler.result:
Bug#30632 test case result
mysql-test/t/handler.test:
Bug#30632 test case
sql/sql_handler.cc:
Always unlock the internal and external table level locks if any of the conditions
(including errors) to read from a HANDLER table are not met.
The problem from a user's perspective: user creates table A, and then tries
to CREATE TABLE a SELECT from A - and this causes a deadlock error, a hang,
or fails with a debug assert, but only if the storage engine is InnoDB.
The origin of the problem: InnoDB uses case-insensitive collation
(system_charset_info) when looking up the internal table share, thus returning
the same share for 'a' and 'A'.
Cause of the user-visible behavior: since the same share is returned to SQL
locking subsystem, it assumes that the same table is first locked (within the
same session) for WRITE, and then for READ, and returns a deadlock error.
However, the code is wrong in not properly cleaning up upon an error, leaving
external locks in place, which leads to assertion failures and hangs.
Fix that has been implemented: the SQL layer should properly propagate the
deadlock error, cleaning up and freeing all resources.
Further work towards a more complete solution: InnoDB should not use case
insensitive collation for table share hash if table names on disk honor the case.
mysql-test/r/innodb-deadlock.result:
Bug#25164 test case result
mysql-test/t/innodb-deadlock.test:
Bug#25164 test case. The CREATE TABLE may fail depending on the character set
of the system and filesystem, but it should never hang.
sql/lock.cc:
Unlock the storage engine "external" table level locks, if the MySQL thr_lock
locking subsystem detects a deadlock error.
into gleb.loc:/home/uchum/work/bk/5.1-opt
mysql-test/r/type_bit.result:
Auto merged
mysql-test/t/type_bit.test:
Auto merged
sql/sql_select.cc:
Merge with 5.0-opt.
into a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-5.1-marvel
client/mysqldump.c:
Auto merged
libmysql/libmysql.c:
Auto merged
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/r/status.result:
Auto merged
sql/event_db_repository.cc:
Auto merged
sql/event_queue.cc:
Auto merged
sql/field.cc:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/handler.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/opt_range.cc:
Auto merged
sql/sp.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/sql_string.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/table.h:
Auto merged
tests/mysql_client_test.c:
Auto merged
mysql-test/t/status.test:
Manual resolve with 5.1.
sql/log_event.cc:
Manual resolve with 5.1.
sql/sql_yacc.yy:
Manual resolve with 5.1.
This test case uses the wait_condition helper (only available in 5.1) in order to wait till the select/update opens and locks the table.
mysql-test/r/sp.result:
Test case result for bug 29936
mysql-test/t/sp.test:
Test case for bug 29936
into a88-113-38-195.elisa-laajakaista.fi:/home/my/bk/mysql-5.1-marvel
libmysql/libmysql.c:
Auto merged
mysql-test/r/query_cache.result:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/handler.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/opt_range.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
tests/mysql_client_test.c:
Auto merged
into pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-5.1-build
libmysql/libmysql.c:
Auto merged
mysql-test/r/information_schema.result:
Auto merged
mysql-test/t/information_schema.test:
Auto merged
sql/ha_partition.cc:
Auto merged
sql/ha_partition.h:
Auto merged
sql/handler.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_cmpfunc.h:
Auto merged
sql/opt_range.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
tests/mysql_client_test.c:
Auto merged
Recommit to 5.1.22.
The bug caused memory corruption for some queries with top OR level
in the WHERE condition if they contained equality predicates and
other sargable predicates in disjunctive parts of the condition.
The corruption happened because the upper bound of the memory
allocated for KEY_FIELD and SARGABLE_PARAM internal structures
containing info about potential lookup keys was calculated incorrectly
in some cases. In particular it was calculated incorrectly when the
WHERE condition was an OR formula with disjuncts being AND formulas
including equalities and other sargable predicates.
mysql-test/r/select.result:
Added a test case for bug #30396.
Recommit to 5.1.22.
mysql-test/t/select.test:
Added a test case for bug #30396.
Recommit to 5.1.22.
sql/item_cmpfunc.h:
Removed max_members from the COND_EQUAL class as not useful anymore.
Recommit to 5.1.22.
sql/sql_base.cc:
Added the max_equal_elems field to the st_select_lex structure.
Recommit to 5.1.22.
sql/sql_lex.cc:
Added the max_equal_elems field to the st_select_lex structure.
Recommit to 5.1.22.
sql/sql_lex.h:
Added the max_equal_elems field to the st_select_lex structure.
The field contains the maximal number of elements in multiple equalities
built for the query conditions.
Recommit to 5.1.22.
sql/sql_select.cc:
Fixed bug #30396.
Recommit to 5.1.22.
The bug caused memory corruption for some queries with top OR level
in the WHERE condition if they contained equality predicates and
other sargable predicates in disjunctive parts of the condition.
The corruption happened because the upper bound of the memory
allocated for KEY_FIELD and SARGABLE_PARAM internal structures
containing info about potential lookup keys was calculated incorrectly
in some cases. In particular it was calculated incorrectly when the
WHERE condition was an OR formula with disjuncts being AND formulas
including equalities and other sargable predicates.
The max_equal_elems field to the st_select_lex structure is used now
to calculate the above mentioned upper bound. The field contains the
maximal number of elements in multiple equalities built for the query
conditions.
into bk-internal.mysql.com:/users/gshchepa/mysql-5.1-opt
sql/item.cc:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_cache.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
Test case contains possible race conditions. This patch fixes these race
conditions and also adjust the test to execute according to its documentation.
mysql-test/r/events.result:
Modified test case to fit test documentation
mysql-test/t/events.test:
Modified test case to fit test documentation
into whalegate.ndb.mysql.com:/home/tomas/mysql-5.1-new-ndb-merge
sql/field.cc:
Auto merged
sql/log_event.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/set_var.cc:
Auto merged
into weblab.(none):/home/marcsql/TREE/mysql-5.1-rt50-merge
BUILD/SETUP.sh:
Auto merged
mysql-test/r/query_cache.result:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
sql/item_cmpfunc.h:
Auto merged
This is a performance bug, related to the parsing or 'OR' and 'AND' boolean
expressions.
Let N be the number of expressions involved in a OR (respectively AND).
When N=1
For example, "select 1" involve only 1 term: there is no OR operator.
In 4.0 and 4.1, parsing expressions not involving OR had no overhead.
In 5.0, parsing adds some overhead, with Select->expr_list.
With this patch, the overhead introduced in 5.0 has been removed,
so that performances for N=1 should be identical to the 4.0 performances,
which are optimal (there is no code executed at all)
The overhead in 5.0 was in fact affecting significantly some operations.
For example, loading 1 Million rows into a table with INSERTs,
for a table that has 100 columns, leads to parsing 100 Millions of
expressions, which means that the overhead related to Select->expr_list
is executed 100 Million times ...
Considering that N=1 is by far the most probable expression,
this case should be optimal.
When N=2
For example, "select a OR b" involves 2 terms in the OR operator.
In 4.0 and 4.1, parsing expressions involving 2 terms created 1 Item_cond_or
node, which is the expected result.
In 5.0, parsing these expression also produced 1 node, but with some extra
overhead related to Select->expr_list : creating 1 list in Select->expr_list
and another in Item_cond::list is inefficient.
With this patch, the overhead introduced in 5.0 has been removed
so that performances for N=2 should be identical to the 4.0 performances.
Note that the memory allocation uses the new (thd->mem_root) syntax
directly.
The cost of "is_cond_or" is estimated to be neglectable: the real problem
of the performance degradation comes from unneeded memory allocations.
When N>=3
For example, "select a OR b OR c ...", which involves 3 or more terms.
In 4.0 and 4.1, the parser had no significant cost overhead, but produced
an Item tree which is difficult to evaluate / optimize during runtime.
In 5.0, the parser produces a better Item tree, using the Item_cond
constructor that accepts a list of children directly, but at an extra cost
related to Select->expr_list.
With this patch, the code is implemented to take the best of the two
implementations:
- there is no overhead with Select->expr_list
- the Item tree generated is optimized and flattened.
This is achieved by adding children nodes into the Item tree directly,
with Item_cond::add(), which avoids the need for temporary lists and memory
allocation
Note that this patch also provide an extra optimization, that the previous
code in 5.0 did not provide: expressions are flattened in the Item tree,
based on what the expression already parsed is, and not based on the order
in which rules are reduced.
For example : "(a OR b) OR c", "a OR (b OR c)" would both be represented
with 2 Item_cond_or nodes before this patch, and with 1 node only with this
patch. The logic used is based on the mathematical properties of the OR
operator (it's associative), and produces a simpler tree.
sql/item_cmpfunc.h:
Improved performances for parsing boolean expressions
sql/sql_yacc.yy:
Improved performances for parsing boolean expressions
mysql-test/r/parser_precedence.result:
Added test cases to cover boolean operator precedence
mysql-test/t/parser_precedence.test:
Added test cases to cover boolean operator precedence
client/mysqldump.c:
Auto merged
sql/event_db_repository.cc:
Auto merged
sql/event_queue.cc:
Auto merged
sql/field.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/table.h:
Auto merged
sql/sql_yacc.yy:
Manual merge.
into hynda.mysql.fi:/home/my/mysql-5.1-marvel
client/mysqldump.c:
Auto merged
sql/event_db_repository.cc:
Auto merged
sql/event_queue.cc:
Auto merged
sql/field.cc:
Auto merged
sql/item.cc:
Auto merged
sql/item_subselect.cc:
Auto merged
sql/log_event.h:
Auto merged
sql/sp.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_db.cc:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_plugin.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_show.cc:
Auto merged
sql/table.h:
Auto merged
sql/sql_yacc.yy:
Manual merge with 5.1 main tree.
into trift2.:/MySQL/M51/push-5.1
Includes manual merges.
mysql-test/mysql-test-run.pl:
Auto merged
mysql-test/suite/ndb/r/ndb_dd_basic.result:
Auto merged
sql/field.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_string.cc:
Auto merged
sql/sql_table.cc:
Auto merged
sql/unireg.h:
Auto merged
mysql-test/r/status.result:
Manual merge of two incompatible fixes to test "status".
mysql-test/t/status.test:
Manual merge of two incompatible fixes to test "status".
sql/log_event.cc:
Manual merge.
sql/log_event.h:
Manual merge.
into hynda.mysql.fi:/home/my/mysql-5.1-marvel
mysql-test/mysql-test-run.pl:
Auto merged
sql/field.cc:
Auto merged
sql/handler.cc:
Auto merged
sql/item_func.cc:
Auto merged
sql/log.cc:
Auto merged
sql/log_event_old.cc:
Auto merged
sql/mysql_priv.h:
Auto merged
sql/mysqld.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_handler.cc:
Auto merged
sql/sql_insert.cc:
Auto merged
sql/sql_select.cc:
Auto merged
sql/sql_string.cc:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/r/status.result:
Manual merge with main 5.0 tree.
mysql-test/t/status.test:
Manual merge with main 5.0 tree.
sql/log_event.cc:
Manual merge with main 5.0 tree.
sql/log_event.h:
Manual merge with main 5.0 tree.
into adventure.(none):/home/thek/Development/cpp/mysql-5.0-runtime
mysql-test/r/query_cache.result:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
