This is a performance bug, affecting in particular the bison generated code
for the parser.
Prior to this fix, the grammar used a long chain of reduces to parse an
expression, like:
bit_expr -> bit_term
bit_term -> bit_factor
bit_factor -> value_expr
value_expr -> term
term -> factor
etc
This chain of reduces cause the internal state automaton in the generated
parser to execute more state transitions and more reduces, so that the
generated MySQLParse() function would spend a lot of time looping to execute
all the grammar reductions.
With this patch, the grammar has been reorganized so that rules are more
"flat", limiting the depth of reduces needed to parse <expr>.
Tests have been written to enforce that relative priorities and properties
of operators have not changed while changing the grammar.
See the bug report for performance data.
mysql-test/r/parser_precedence.result:
Improved test coverage for operator precedence
mysql-test/t/parser_precedence.test:
Improved test coverage for operator precedence
sql/sql_yacc.yy:
Simplified the grammar to improve performances
If, after the tables are locked, one of the conditions to read from a
HANDLER table is not met, the handler code wrongly jumps to a error path
that won't unlock the tables.
The user-visible effect is that after a error in a handler read command,
all subsequent handler operations on the same table will hang.
The fix is simply to correct the code to jump to the (same) error path that
unlocks the tables.
mysql-test/r/handler.result:
Bug#30632 test case result
mysql-test/t/handler.test:
Bug#30632 test case
sql/sql_handler.cc:
Always unlock the internal and external table level locks if any of the conditions
(including errors) to read from a HANDLER table are not met.
The problem from a user's perspective: user creates table A, and then tries
to CREATE TABLE a SELECT from A - and this causes a deadlock error, a hang,
or fails with a debug assert, but only if the storage engine is InnoDB.
The origin of the problem: InnoDB uses case-insensitive collation
(system_charset_info) when looking up the internal table share, thus returning
the same share for 'a' and 'A'.
Cause of the user-visible behavior: since the same share is returned to SQL
locking subsystem, it assumes that the same table is first locked (within the
same session) for WRITE, and then for READ, and returns a deadlock error.
However, the code is wrong in not properly cleaning up upon an error, leaving
external locks in place, which leads to assertion failures and hangs.
Fix that has been implemented: the SQL layer should properly propagate the
deadlock error, cleaning up and freeing all resources.
Further work towards a more complete solution: InnoDB should not use case
insensitive collation for table share hash if table names on disk honor the case.
mysql-test/r/innodb-deadlock.result:
Bug#25164 test case result
mysql-test/t/innodb-deadlock.test:
Bug#25164 test case. The CREATE TABLE may fail depending on the character set
of the system and filesystem, but it should never hang.
sql/lock.cc:
Unlock the storage engine "external" table level locks, if the MySQL thr_lock
locking subsystem detects a deadlock error.
This is a performance bug, related to the parsing or 'OR' and 'AND' boolean
expressions.
Let N be the number of expressions involved in a OR (respectively AND).
When N=1
For example, "select 1" involve only 1 term: there is no OR operator.
In 4.0 and 4.1, parsing expressions not involving OR had no overhead.
In 5.0, parsing adds some overhead, with Select->expr_list.
With this patch, the overhead introduced in 5.0 has been removed,
so that performances for N=1 should be identical to the 4.0 performances,
which are optimal (there is no code executed at all)
The overhead in 5.0 was in fact affecting significantly some operations.
For example, loading 1 Million rows into a table with INSERTs,
for a table that has 100 columns, leads to parsing 100 Millions of
expressions, which means that the overhead related to Select->expr_list
is executed 100 Million times ...
Considering that N=1 is by far the most probable expression,
this case should be optimal.
When N=2
For example, "select a OR b" involves 2 terms in the OR operator.
In 4.0 and 4.1, parsing expressions involving 2 terms created 1 Item_cond_or
node, which is the expected result.
In 5.0, parsing these expression also produced 1 node, but with some extra
overhead related to Select->expr_list : creating 1 list in Select->expr_list
and another in Item_cond::list is inefficient.
With this patch, the overhead introduced in 5.0 has been removed
so that performances for N=2 should be identical to the 4.0 performances.
Note that the memory allocation uses the new (thd->mem_root) syntax
directly.
The cost of "is_cond_or" is estimated to be neglectable: the real problem
of the performance degradation comes from unneeded memory allocations.
When N>=3
For example, "select a OR b OR c ...", which involves 3 or more terms.
In 4.0 and 4.1, the parser had no significant cost overhead, but produced
an Item tree which is difficult to evaluate / optimize during runtime.
In 5.0, the parser produces a better Item tree, using the Item_cond
constructor that accepts a list of children directly, but at an extra cost
related to Select->expr_list.
With this patch, the code is implemented to take the best of the two
implementations:
- there is no overhead with Select->expr_list
- the Item tree generated is optimized and flattened.
This is achieved by adding children nodes into the Item tree directly,
with Item_cond::add(), which avoids the need for temporary lists and memory
allocation
Note that this patch also provide an extra optimization, that the previous
code in 5.0 did not provide: expressions are flattened in the Item tree,
based on what the expression already parsed is, and not based on the order
in which rules are reduced.
For example : "(a OR b) OR c", "a OR (b OR c)" would both be represented
with 2 Item_cond_or nodes before this patch, and with 1 node only with this
patch. The logic used is based on the mathematical properties of the OR
operator (it's associative), and produces a simpler tree.
sql/item_cmpfunc.h:
Improved performances for parsing boolean expressions
sql/sql_yacc.yy:
Improved performances for parsing boolean expressions
mysql-test/r/parser_precedence.result:
Added test cases to cover boolean operator precedence
mysql-test/t/parser_precedence.test:
Added test cases to cover boolean operator precedence
into adventure.(none):/home/thek/Development/cpp/mysql-5.0-runtime
mysql-test/r/query_cache.result:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
Although the query cache doesn't support retrieval of statements containing
column level access control, it was still possible to cache such statements
thus wasting memory.
This patch extends the access control check on the target tables to avoid
caching a statement with column level restrictions.
Views are excepted and can be cached but only retrieved by super user account.
mysql-test/t/query_cache_with_views.test:
Rename: mysql-test/t/view_query_cache.test -> mysql-test/t/query_cache_with_views.test
mysql-test/r/query_cache_with_views.result:
Rename: mysql-test/r/view_query_cache.result -> mysql-test/r/query_cache_with_views.result
mysql-test/r/query_cache.result:
Modified test case to allow caching of views
mysql-test/t/query_cache.test:
Modified test case to allow caching of views
sql/sql_cache.cc:
Allow caching of views
into adventure.(none):/home/thek/Development/cpp/mysql-5.0-runtime
mysql-test/r/query_cache.result:
Auto merged
mysql-test/t/query_cache.test:
Auto merged
Although the query cache doesn't support retrieval of statements containing
column level access control, it was still possible to cache such statements
thus wasting memory.
This patch extends the access control check on the target tables to avoid
caching a statement with column level restrictions.
mysql-test/r/query_cache.result:
Added test
mysql-test/t/query_cache.test:
Added test
sql/sql_cache.cc:
The function check_table_access leaves the artifact
grant.want_privileges= 1, if a statement refers to tables with column level
privileges. To avoid the statement from being stored into the query cache,
it is enough to check this flag and set 'safe_to_cache_query' to zero.
sql/sql_cache.h:
- Removed 'static' attribute or class methods
- Added THD parameter to process_and_count_tables
mysql_ha_open calls mysql_ha_close on the error path (unsupported) to close the (opened) table before inserting it into the tables hash list handler_tables_hash) but mysql_ha_close only closes tables which are on the hash list, causing the table to be left open and locked.
This change moves the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close (mysql_ha_close).
mysql-test/r/handler.result:
Bug#25856 test result
mysql-test/t/handler.test:
Bug#25856 test case
sql/sql_handler.cc:
Move the table close logic into a separate function that is always called on the error path of mysql_ha_open or on a normal handler close
into bodhi.(none):/opt/local/work/mysql-5.0-runtime
mysql-test/r/federated.result:
Auto merged
mysql-test/t/federated.test:
Auto merged
sql/item.cc:
Auto merged
under terms of bug#28875 for better performance.
The change appeared to require more changes in item_cmpfunc.cc,
which is dangerous in 5.0.
Conversion between a latin1 column and an ascii string constant
stopped to work.
mysql-test/r/ctype_recoding.result:
Adding test case.
mysql-test/t/ctype_recoding.test:
Adding test case.
Two character mappings were way off (backtick and tilde were "E"
and "Y"!), and three others were slightly rotated. The first
would cause collisions, and the latter was probably benign.
Now, assign the character mappings exactly to their normal values.
sql/share/charsets/ascii.xml:
Change the character mapping for
"`" to "`" (was "E") and
"[" to "[" (was "\") and
"\" to "\" (was "]") and
"]" to "]" (was "[") and
"~" to "~" (was "Y").
strings/ctype-extra.c:
Generated from charsets directory.
mysql-test/r/ctype_ascii.result:
Add new test file.
Test all combinations of printable letter comparisons for
similarity.
mysql-test/t/ctype_ascii.test:
Add new test file.
Test all combinations of printable letter comparisons for
similarity.
MySQL replicates the time zone only when operations that involve
it are performed. This is controlled by a flag. But this flag
is set only on successful operation.
The flag must be set also when there is an error that involves
a timezone (so the master would replicate the error to the slaves).
Fixed by moving the setting of the flag before the operation
(so it apples to errors as well).
mysql-test/r/rpl_timezone.result:
Bug #29536: test case
mysql-test/t/rpl_timezone.test:
Bug #29536: test case
sql/field.cc:
Bug #29536: move setting of the flag before the operation
(so it apples to errors as well).
sql/time.cc:
Bug #29536: move setting of the flag before the operation
(so it apples to errors as well).
Search "relwithdebinfo" directory in CMake Visual Studio build
Search for "mysqld-debug" even in source tree
mysql-test/mysql-test-run.pl:
Search "relwithdebinfo" directory in CMake Visual Studio build
Search for "mysqld-debug" even in source tree
This bug manifested itself for queries with grouping by columns of
the BIT type. It led to wrong comparisons of bit-field values and
wrong result sets.
Bit-field values never cannot be compared as binary values. Yet
the class Field_bit had an implementation of the cmp method that
compared bit-fields values as binary values.
Also the get_image and set_image methods of the base class Field
cannot be used for objects of the Field_bit class.
Now these methods are declared as virtual and specific implementations
of the methods are provided for the class Field_bit.
mysql-test/r/type_bit.result:
Added a test case for bug #30219.
mysql-test/t/type_bit.test:
Added a test case for bug #30219.
sql/field.h:
Fix bug #30219.
This bug manifested itself for queries with grouping by columns of
the BIT type. It led to wrong comparisons of bit-field values and
wrong result sets.
Bit-field values never cannot be compared as binary values. Yet
the class Field_bit had an implementation of the cmp method that
compared bit-fields values as binary values.
Also the get_image and set_image methods of the base class Field
cannot be used for objects of the Field_bit class.
Now these methods are declared as virtual and specific implementations
of these methods are provided for the class Field_bit.
into dl145h.mysql.com:/data0/mkindahl/mysql-5.0-2team
mysql-test/r/func_time.result:
Auto merged
mysql-test/t/func_time.test:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
A test case was waiting for a fixed number of seconds for a specific
state of the slave IO thread to take place.
Fixed by waiting in a loop for that specific thread state instead
(or timeout).
mysql-test/t/rpl_relayspace.test:
Bug #25228: fixed test case
into mysql.com:/home/bar/mysql-work/mysql-5.0-rpl
mysql-test/r/ctype_utf8.result:
Auto merged
mysql-test/t/ctype_utf8.test:
Auto merged
sql/item.cc:
Auto merged
sql/item.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/item_strfunc.cc:
Auto merged
sql/item_timefunc.cc:
Auto merged
sql/sql_lex.cc:
Auto merged
sql/sql_lex.h:
Auto merged
sql/sql_yacc.yy:
Auto merged
strings/conf_to_src.c:
Auto merged
strings/ctype-extra.c:
Auto merged
mysql-test/r/ctype_ucs.result:
After merge fix
mysql-test/t/ctype_ucs.test:
After merge fix
(Regression, caused by a patch for the bug 22646).
Problem: when result type of date_format() was changed from
binary string to character string, mixing date_format()
with a ascii column in CONCAT() stopped to work.
Fix:
- adding "repertoire" flag into DTCollation class,
to mark items which can return only pure ASCII strings.
- allow character set conversion from pure ASCII to other character sets.
include/m_ctype.h:
Defining new flags.
Adding new function prototypes.
mysql-test/r/ctype_ucs.result:
Adding tests.
mysql-test/r/ctype_utf8.result:
Adding tests.
mysql-test/r/func_time.result:
Adding tests.
mysql-test/t/ctype_ucs.test:
Adding tests.
mysql-test/t/ctype_utf8.test:
Adding tests.
mysql-test/t/func_time.test:
Adding test.
mysys/charset.c:
Adding pure ASCII detection when loading a dynamic character set.
sql/item.cc:
- Moving detection of a Unicode superset into function.
- Adding detection of a ASCII subset.
- Adding creation of to-ASCII character set convertor when
safe_charset_converter() failed and when the argument.
repertoire is know to be pure ASCII.
sql/item.h:
- Adding "repertoire" member into DTCollation class.
- Adding "repertoire" argument to constructors.
- Adding new methods:
set_repertoire_from_charset()
set_repertoire_from_value()
sql/item_func.cc:
Adding "repertoire" argument.
sql/item_strfunc.cc:
Adding "repertoire" argument.
sql/item_timefunc.cc:
Initializing the result repertoire taking into account the "is_ascii"
flag of the current locale.
sql/sql_lex.cc:
Detect 7bit strings, return in Lex->text_string_is_7bit.
sql/sql_lex.h:
Adding new member into LEX structure.
Adding new member into Lex_input_stream
sql/sql_string.cc:
Allow simple copy from pure ASCII to a ASCII-based character set.
sql/sql_yacc.yy:
Depening on Lex->text_string_is_7bit and character set features,
create Item_string with MY_REPERTOIRE_ASCII when it is possible.
strings/conf_to_src.c:
- Adding printing of the "MY_CS_PUREASCII" flag
- Adding printing of copyright
strings/ctype-extra.c:
Recreating ctype-extra.c: ascii_general_ci and ascii_bin
are now marked with MY_CS_PUREASCII flag.
strings/ctype.c:
Adding new functions.
The SELECT query with more than 31 nested dependent SELECT queries returned
wrong result.
New error message has been added: ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT.
It will be reported as: "Too high level of nesting for select".
sql/sql_parse.cc:
Fixed bug #27352.
The Item_sum::register_sum_func method has been modified to return
TRUE on exceeding of allowed level of SELECT nesting and to report
corresponding error message.
sql/unireg.h:
Fixed bug #27352.
Constant definition has been added: maximal allowed level of SELECT nesting.
mysql-test/t/select.test:
Updated test case for bug #27352.
mysql-test/r/select.result:
Updated test case for bug #27352.
sql/share/errmsg.txt:
Fixed bug #27352.
New error message has been added: ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT.
into ramayana.hindu.god:/home/tsmith/m/bk/maint/50
sql/handler.h:
Auto merged
sql/sql_base.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_table.cc:
Auto merged
sql/sql_yacc.yy:
Auto merged
sql/table.cc:
Auto merged
into ramayana.hindu.god:/home/tsmith/m/bk/maint/50
mysql-test/r/federated.result:
Auto merged
mysql-test/t/federated.test:
Auto merged
sql/ha_federated.h:
Auto merged
sql/item_func.cc:
Auto merged
sql/sql_class.h:
Auto merged
sql/sql_table.cc:
Auto merged
mysql-test/r/innodb_mysql.result:
Manual merge
mysql-test/t/innodb_mysql.test:
Manual merge
error 1159 ' ' (fixed by the patch for Bug 25679)
mysql-test/r/federated.result:
Update results (Bug#18287)
mysql-test/t/federated.test:
Add a test case for Bug#18287 create federated table always times out,
error 1159 ' '
restores from mysqlbinlog out
Problem: using "mysqlbinlog | mysql" for recoveries the connection_id()
result may differ from what was used when issuing the statement.
Fix: if there is a connection_id() in a statement, write to binlog
SET pseudo_thread_id= XXX; before it and use the value later on.
mysql-test/r/mysqlbinlog.result:
Fix for bug #29928: INSERT ... VALUES(connection_id(), ...) incorrect
restores from mysqlbinlog out
- test result.
mysql-test/t/mysqlbinlog.test:
Fix for bug #29928: INSERT ... VALUES(connection_id(), ...) incorrect
restores from mysqlbinlog out
- test case.
sql/item_create.cc:
Fix for bug #29928: INSERT ... VALUES(connection_id(), ...) incorrect
restores from mysqlbinlog out
- set thread_specific_used flag for the connection_id() function.
sql/item_func.cc:
Fix for bug #29928: INSERT ... VALUES(connection_id(), ...) incorrect
restores from mysqlbinlog out
- always return thd->variables.pseudo_thread_id as a connection_id()
result, as it contains a proper value for both master and slave.
sql/log_event.cc:
Fix for bug #29928: INSERT ... VALUES(connection_id(), ...) incorrect
restores from mysqlbinlog out
- set LOG_EVENT_THREAD_SPECIFIC_F event flag if thread_specific_used
is set.
sql/sql_class.cc:
Fix for bug #29928: INSERT ... VALUES(connection_id(), ...) incorrect
restores from mysqlbinlog out
- thd->thread_specific_used introduced, which is set if thread specific
value(s) used in a statement.
sql/sql_class.h:
Fix for bug #29928: INSERT ... VALUES(connection_id(), ...) incorrect
restores from mysqlbinlog out
- thd->thread_specific_used introduced, which is set if thread specific
value(s) used in a statement.
mysql-test/r/func_time.result:
Update results (use fixed datetime values instead of NOW()).
mysql-test/t/func_time.test:
Use fixed datetime values instead of NOW(): the test would have a sporadic
failure when current day changed between two consequtive calls to
NOW(). The test actually tests FROM_DAYS/TO_DAYS functions,
so use of NOW() is not necessary.
- test result.
Problem: we don't proper handle "" value of the --myisam-recover option.
Fix: turn off myisam recovering if --myisam-recover="" is set.
mysql-test/r/show_check.result:
Fix for bug #30088: Can't disable myisam-recover by a value of "".
- test result.
mysql-test/t/show_check-master.opt:
Fix for bug #30088: Can't disable myisam-recover by a value of "".
- test case.
mysql-test/t/show_check.test:
Fix for bug #30088: Can't disable myisam-recover by a value of "".
- test case.
sql/mysqld.cc:
Fix for bug #30088: Can't disable myisam-recover by a value of "".
- turn off myisam recovering if --myisam-recover="" is passed.
into bodhi.(none):/opt/local/work/mysql-5.0-runtime
client/mysqldump.c:
Auto merged
mysql-test/r/mysqldump.result:
Auto merged
sql/sql_base.cc:
Auto merged
sql/table.cc:
Auto merged
mysql-test/r/innodb_mysql.result:
Manual merge.
mysql-test/t/innodb_mysql.test:
Manual merge.
sql/sql_table.cc:
Manual merge.
into magare.gmz:/home/kgeorge/mysql/autopush/B27417-5.0-opt
sql/handler.cc:
Auto merged
sql/sp_head.cc:
Auto merged
sql/sql_class.cc:
Auto merged
sql/sql_class.h:
Auto merged
