For queries with order by clauses that employed filesort usage of
virtual column references in select lists could trigger assertion
failures. It happened because a wrong vcol_set bitmap was used for
filesort. It turned out that filesort required its own vcol_set bitmap.
Made management of the vcol_set bitmaps similar to the management
of the read_set and write_set bitmaps.
There were two problems that caused wrong results reported with this bug.
1. In some cases stored(persistent) virtual columns were not marked
in the write_set and in the vcol_set bitmaps.
2. If the list of fields in an insert command was empty then the values of
the stored virtual columns were set to default.
To fix the first problem the function st_table::mark_virtual_columns_for_write
was modified. Now the function has a parameter that says whether the virtual
columns are to be marked for insert or for update.
To fix the second problem a special handling of empty insert lists is
added in the function fill_record().
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h
Conflicts:
Text conflict in mysql-test/r/archive.result
Contents conflict in mysql-test/r/innodb_bug38231.result
Text conflict in mysql-test/r/mdl_sync.result
Text conflict in mysql-test/suite/binlog/t/disabled.def
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result
Text conflict in mysql-test/t/archive.test
Contents conflict in mysql-test/t/innodb_bug38231.test
Text conflict in mysql-test/t/mdl_sync.test
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_show.cc
Text conflict in sql/table.cc
Text conflict in sql/table.h
transactional SELECT and ALTER TABLE ... REBUILD PARTITION".
Make open flags part of Open_table_context.
This allows to simplify some code and (in future)
enforce the invariant that we don't, say, request a back
off on the table when there is MYSQL_OPEN_IGNORE_FLUSH
flag.
sql/sql_base.cc:
open_table() flags are part of Open_table_context.
Remove dead code that would check for OPEN_VIEW_NO_PARSE,
which is not an open table flag.
sql/sql_base.h:
Move flags to Open_table_context. Reorder Open_table_context
members to compact the structure footprint.
sql/sql_insert.cc:
Update with a new calling signature of open_table().
sql/sql_table.cc:
Update with a new calling signature of open_table().
transactional SELECT and ALTER TABLE ... REBUILD PARTITION".
Make open flags part of Open_table_context.
This allows to simplify some code and (in future)
enforce the invariant that we don't, say, request a back
off on the table when there is MYSQL_OPEN_IGNORE_FLUSH
flag.
for ALTER TABLE, LOAD DATA).
ROW_COUNT is now assigned according to the following rules:
- In my_ok():
- for DML statements: to the number of affected rows;
- for DDL statements: to 0.
- In my_eof(): to -1 to indicate that there was a result set.
We derive this semantics from the JDBC specification, where int
java.sql.Statement.getUpdateCount() is defined to (sic) "return the
current result as an update count; if the result is a ResultSet
object or there are no more results, -1 is returned".
- In my_error(): to -1 to be compatible with the MySQL C API and
MySQL ODBC driver.
- For SIGNAL statements: to 0 per WL#2110 specification. Zero is used
since that's the "default" value of ROW_COUNT in the diagnostics area.
sql/protocol.cc:
Fix a typo.
sql/sql_class.h:
- Introduce THD::get_row_count_func() / THD::set_row_count_func();
- Remove the CF_HAS_ROW_COUNT define
sql/sql_parse.cc:
CF_HAS_ROW_COUNT was eliminated.
for ALTER TABLE, LOAD DATA).
ROW_COUNT is now assigned according to the following rules:
- In my_ok():
- for DML statements: to the number of affected rows;
- for DDL statements: to 0.
- In my_eof(): to -1 to indicate that there was a result set.
We derive this semantics from the JDBC specification, where int
java.sql.Statement.getUpdateCount() is defined to (sic) "return the
current result as an update count; if the result is a ResultSet
object or there are no more results, -1 is returned".
- In my_error(): to -1 to be compatible with the MySQL C API and
MySQL ODBC driver.
- For SIGNAL statements: to 0 per WL#2110 specification. Zero is used
since that's the "default" value of ROW_COUNT in the diagnostics area.
Docs/sp-imp-spec.txt:
New sql_mode added.
include/my_base.h:
Flag in frm of create options.
libmysqld/CMakeLists.txt:
New files added.
libmysqld/Makefile.am:
New files added.
mysql-test/r/events_bugs.result:
New sql_mode added.
mysql-test/r/information_schema.result:
New sql_mode added.
mysql-test/r/sp.result:
New sql_mode added.
mysql-test/r/system_mysql_db.result:
New sql_mode added.
mysql-test/suite/funcs_1/r/is_columns_mysql.result:
New sql_mode added.
mysql-test/suite/funcs_1/r/is_columns_mysql_embedded.result:
New sql_mode added.
mysql-test/t/events_bugs.test:
New sql_mode added.
mysql-test/t/sp.test:
New sql_mode added.
scripts/mysql_system_tables.sql:
New sql_mode added.
scripts/mysql_system_tables_fix.sql:
New sql_mode added.
sql/CMakeLists.txt:
New files added.
sql/Makefile.am:
New files added.
sql/event_db_repository.cc:
New sql_mode added.
sql/field.cc:
Create options support added.
sql/field.h:
Create options support added.
sql/ha_partition.cc:
Create options support added.
sql/handler.cc:
Create options support added.
sql/handler.h:
Create options support added.
sql/log_event.h:
New sql_mode added.
sql/mysql_priv.h:
New sql_mode added.
sql/mysqld.cc:
New sql_mode added.
sql/share/errmsg.txt:
New error messages added.
sql/sp.cc:
New sql_mode added.
sql/sp_head.cc:
Create options support added.
sql/sql_class.cc:
Create options support added.
Debug added.
sql/sql_class.h:
Create options support added.
sql/sql_insert.cc:
my_safe_a* moved to mysqld_priv.h
sql/sql_lex.h:
Create options support added.
sql/sql_parse.cc:
Create options support added.
sql/sql_show.cc:
Create options support added.
sql/sql_table.cc:
Create options support added.
sql/sql_view.cc:
New sql_mode added.
sql/sql_yacc.yy:
Create options support added.
sql/structs.h:
Create options support added.
sql/table.cc:
Create options support added.
sql/table.h:
Create options support added.
sql/unireg.cc:
Create options support added.
storage/example/ha_example.cc:
Create options example.
storage/example/ha_example.h:
Create options example.
storage/pbxt/src/discover_xt.cc:
Create options support added.
Adding my_global.h first in all files using
NO_EMBEDDED_ACCESS_CHECKS.
Correcting a merge problem resulting from a changed definition
of check_some_access compared to the original patches.
Adding my_global.h first in all files using
NO_EMBEDDED_ACCESS_CHECKS.
Correcting a merge problem resulting from a changed definition
of check_some_access compared to the original patches.
Conflicts:
Text conflict in mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
Text conflict in sql/log.cc
Text conflict in sql/set_var.cc
Text conflict in sql/sql_class.cc
Conflicts:
Text conflict in mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
Text conflict in sql/log.cc
Text conflict in sql/set_var.cc
Text conflict in sql/sql_class.cc
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
Makefile.am:
add new API files to the check_abi rule,
remove duplicates
client/CMakeLists.txt:
now a client can use dlopen too
client/Makefile.am:
be csh-friendly
include/my_global.h:
add dummy plugs for dlopen and co.
for the code that needs them to work in static builds
mysys/Makefile.am:
be csh-friendly
plugin/auth/dialog.c:
typo fixed
CHECK_FIELD_IGNORE was treated as CHECK_FIELD_ERROR_FOR_NULL;
UPDATE...SET...NULL on NOT NULL fields behaved differently after
a trigger.
Now distinguishes between IGNORE and ERROR_FOR_NULL and save/restores
check-field options.
mysql-test/r/trigger.result:
Show that UPDATE...SET...NULL on NOT NULL columns doesn't behave differently
when run after a trigger.
mysql-test/t/trigger.test:
Show that UPDATE...SET...NULL on NOT NULL columns doesn't behave differently
when run after a trigger.
sql/field_conv.cc:
CHECK_FIELD_IGNORE was treated as CHECK_FIELD_ERROR_FOR_NULL.
Distinguish between the two.
sql/sp_head.cc:
Raise error as needed.
sql/sql_class.cc:
Save and restore check-fields options.
sql/sql_class.h:
Make room so we can save check-fields options.
sql/sql_insert.cc:
Raise error as needed.
CHECK_FIELD_IGNORE was treated as CHECK_FIELD_ERROR_FOR_NULL;
UPDATE...SET...NULL on NOT NULL fields behaved differently after
a trigger.
Now distinguishes between IGNORE and ERROR_FOR_NULL and save/restores
check-field options.
DDL no longer aborts mysql_lock_tables(), and hence
we no longer need to support need_reopen flag of this
call.
Remove the flag, and all the code in the server
that was responsible for handling the case when
it was set. This allowed to simplify:
open_and_lock_tables_derived(), the delayed thread,
multi-update.
Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH,
since we now only support this flag in open_table().
Rename MYSQL_LOCK_PERF_SCHEMA to MYSQL_LOCK_LOG_TABLE,
to avoid confusion.
Move the wait for the global read lock for cases
when we do updates in SELECT f1() or DO (UPDATE) to
open_table() from mysql_lock_tables(). When waiting
for the read lock, we could raise need_reopen flag,
which is no longer present in mysql_lock_tables().
Since the block responsible for waiting for GRL
was moved, MYSQL_LOCK_IGNORE_GLOBAL_READ_LOCK
was renamed to MYSQL_OPEN_IGNORE_GLOBAL_READ_LOCK.
mysql-test/r/mdl_sync.result:
Update test results (see comments for mdl_sync.test).
mysql-test/t/mdl_sync.test:
Update tests: an abort mysql_lock_tables() called for an
INSERT no longer auto-closes SQL HANDLERS, since it
no longer leads to back-off and retry.
sql/ha_ndbcluster_binlog.cc:
Remove unused variables.
sql/lock.cc:
Remove support for need_reopen parameter of mysql_lock_tables().
Update comments.
sql/log_event_old.cc:
Remove the loop responsible for handling need_reopen
out parameter of mysql_lock_tables().
sql/mysql_priv.h:
Update open and lock tables flag names.
sql/share/errmsg-utf8.txt:
Add a new error message to report when
thr_multi_lock() is aborted.
sql/sql_base.cc:
Update comments. Rename MYSQL_LOCK_IGNORE_FLUSH
to MYSQL_OPEN_IGNORE_FLUSH.
sql/sql_class.h:
Remove unused code.
sql/sql_db.cc:
Remove an unused bit of code.
sql/sql_handler.cc:
For backward compatibility, we still want to back off and
retry when a call to mysql_lock_tables() is aborted
from within an SQL HANDLER. Write an internal error
handler to support the case.
sql/sql_insert.cc:
Call mysql_lock_tables() no longer has need_reopen
out parameter. Simplify the code by removing
the crud that took care of it.
MYSQL_LOCK_IGNORE_FLUSH is now only supported by
open_tables().
sql/sql_show.cc:
Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH
sql/sql_table.cc:
Remove an unused parameter.
sql/sql_update.cc:
Remove the need_reopen loop from multi-update.
We no also longer need to cleanup the parse tree in case
when mysql_lock_tables() is aborted and thus an infinite
source of multi-update bugs is gone.
sql/tztime.cc:
Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH,
since from now on this flag is only supported by open_table().
DDL no longer aborts mysql_lock_tables(), and hence
we no longer need to support need_reopen flag of this
call.
Remove the flag, and all the code in the server
that was responsible for handling the case when
it was set. This allowed to simplify:
open_and_lock_tables_derived(), the delayed thread,
multi-update.
Rename MYSQL_LOCK_IGNORE_FLUSH to MYSQL_OPEN_IGNORE_FLUSH,
since we now only support this flag in open_table().
Rename MYSQL_LOCK_PERF_SCHEMA to MYSQL_LOCK_LOG_TABLE,
to avoid confusion.
Move the wait for the global read lock for cases
when we do updates in SELECT f1() or DO (UPDATE) to
open_table() from mysql_lock_tables(). When waiting
for the read lock, we could raise need_reopen flag,
which is no longer present in mysql_lock_tables().
Since the block responsible for waiting for GRL
was moved, MYSQL_LOCK_IGNORE_GLOBAL_READ_LOCK
was renamed to MYSQL_OPEN_IGNORE_GLOBAL_READ_LOCK.
This patch prevents system threads and system table accesses from
using user-specified values for "lock_wait_timeout". Instead all
such accesses are done using the default value (1 year).
This prevents background tasks (such as replication, events,
accessing stored function definitions, logging, reading time-zone
information, etc.) from failing in cases where the global value
of "lock_wait_timeout" is set very low.
The patch also simplifies the open tables API. Rather than adding
another convenience function for opening and locking system tables,
this patch removes most of the existing convenience functions for
open_and_lock_tables_derived(). Before, open_and_lock_tables() was
a convenience function that enforced derived tables handling, while
open_and_lock_tables_derived() was the main function where derived
tables handling was optional. Now, this convencience function is
gone and the main function is renamed to open_and_lock_tables().
No test case added as it would have required the use of --sleep to
check that system threads and system tables have a different timeout
value from the user-specified "lock_wait_timeout" system variable.
This patch prevents system threads and system table accesses from
using user-specified values for "lock_wait_timeout". Instead all
such accesses are done using the default value (1 year).
This prevents background tasks (such as replication, events,
accessing stored function definitions, logging, reading time-zone
information, etc.) from failing in cases where the global value
of "lock_wait_timeout" is set very low.
The patch also simplifies the open tables API. Rather than adding
another convenience function for opening and locking system tables,
this patch removes most of the existing convenience functions for
open_and_lock_tables_derived(). Before, open_and_lock_tables() was
a convenience function that enforced derived tables handling, while
open_and_lock_tables_derived() was the main function where derived
tables handling was optional. Now, this convencience function is
gone and the main function is renamed to open_and_lock_tables().
No test case added as it would have required the use of --sleep to
check that system threads and system tables have a different timeout
value from the user-specified "lock_wait_timeout" system variable.
CHECK_FIELD_IGNORE was treated as CHECK_FIELD_ERROR_FOR_NULL;
UPDATE...SET...NULL on NOT NULL fields behaved differently after
a trigger.
Now distinguishes between IGNORE and ERROR_FOR_NULL and save/restores
check-field options.
mysql-test/r/trigger.result:
Show that UPDATE...SET...NULL on NOT NULL columns doesn't behave differently
when run after a trigger.
mysql-test/t/trigger.test:
Show that UPDATE...SET...NULL on NOT NULL columns doesn't behave differently
when run after a trigger.
sql/field_conv.cc:
CHECK_FIELD_IGNORE was treated as CHECK_FIELD_ERROR_FOR_NULL.
Distinguish between the two.
sql/sp_head.cc:
raise error as needed
sql/sql_class.cc:
Save and restore check-fields options.
sql/sql_class.h:
Make room so we can save check-fields options.
sql/sql_insert.cc:
raise error as needed
CHECK_FIELD_IGNORE was treated as CHECK_FIELD_ERROR_FOR_NULL;
UPDATE...SET...NULL on NOT NULL fields behaved differently after
a trigger.
Now distinguishes between IGNORE and ERROR_FOR_NULL and save/restores
check-field options.
This patch introduces timeouts for metadata locks.
The timeout is specified in seconds using the new dynamic system
variable "lock_wait_timeout" which has both GLOBAL and SESSION
scopes. Allowed values range from 1 to 31536000 seconds (= 1 year).
The default value is 1 year.
The new server parameter "lock-wait-timeout" can be used to set
the default value parameter upon server startup.
"lock_wait_timeout" applies to all statements that use metadata locks.
These include DML and DDL operations on tables, views, stored procedures
and stored functions. They also include LOCK TABLES, FLUSH TABLES WITH
READ LOCK and HANDLER statements.
The patch also changes thr_lock.c code (table data locks used by MyISAM
and other simplistic engines) to use the same system variable.
InnoDB row locks are unaffected.
One exception to the handling of the "lock_wait_timeout" variable
is delayed inserts. All delayed inserts are executed with a timeout
of 1 year regardless of the setting for the global variable. As the
connection issuing the delayed insert gets no notification of
delayed insert timeouts, we want to avoid unnecessary timeouts.
It's important to note that the timeout value is used for each lock
acquired and that one statement can take more than one lock.
A statement can therefore block for longer than the lock_wait_timeout
value before reporting a timeout error. When lock timeout occurs,
ER_LOCK_WAIT_TIMEOUT is reported.
Test case added to lock_multi.test.
include/my_pthread.h:
Added macros for comparing two timespec structs.
include/thr_lock.h:
Introduced timeouts for thr_lock.c locks.
mysql-test/r/mysqld--help-notwin.result:
Updated result file with the new server variable.
mysql-test/r/mysqld--help-win.result:
Updated result file with the new server variable.
mysql-test/suite/sys_vars/r/lock_wait_timeout_basic.result:
Added basic test for the new server variable.
mysql-test/suite/sys_vars/t/lock_wait_timeout_basic.test:
Added basic test for the new server variable.
mysys/thr_lock.c:
Introduced timeouts for thr_lock.c locks.
sql/mdl.cc:
Introduced timeouts for metadata locks.
sql/mdl.h:
Introduced timeouts for metadata locks.
sql/sql_base.cc:
Introduced timeouts in tdc_wait_for_old_versions().
sql/sql_class.h:
Added new server variable lock_wait_timeout.
sql/sys_vars.cc:
Added new server variable lock_wait_timeout.
This patch introduces timeouts for metadata locks.
The timeout is specified in seconds using the new dynamic system
variable "lock_wait_timeout" which has both GLOBAL and SESSION
scopes. Allowed values range from 1 to 31536000 seconds (= 1 year).
The default value is 1 year.
The new server parameter "lock-wait-timeout" can be used to set
the default value parameter upon server startup.
"lock_wait_timeout" applies to all statements that use metadata locks.
These include DML and DDL operations on tables, views, stored procedures
and stored functions. They also include LOCK TABLES, FLUSH TABLES WITH
READ LOCK and HANDLER statements.
The patch also changes thr_lock.c code (table data locks used by MyISAM
and other simplistic engines) to use the same system variable.
InnoDB row locks are unaffected.
One exception to the handling of the "lock_wait_timeout" variable
is delayed inserts. All delayed inserts are executed with a timeout
of 1 year regardless of the setting for the global variable. As the
connection issuing the delayed insert gets no notification of
delayed insert timeouts, we want to avoid unnecessary timeouts.
It's important to note that the timeout value is used for each lock
acquired and that one statement can take more than one lock.
A statement can therefore block for longer than the lock_wait_timeout
value before reporting a timeout error. When lock timeout occurs,
ER_LOCK_WAIT_TIMEOUT is reported.
Test case added to lock_multi.test.
mysql-test/t/disabled.def:
Restore disabled ssl tests: SSL certificates were updated.
Disable sp_sync.test, the test case can't work in next-4284.
mysql-test/t/partition_innodb.test:
Disable parsing of the test case for Bug#47343,
the test can not work in next-4284.
mysql-test/t/ps_ddl.test:
Update results (CREATE TABLE IF NOT EXISTS takes
into account existence of the temporary table).
