DISABLE AND ENABLED DURING DDL OPERATION
PROBLEM: Same thread trying to acquire the same mutex
second time leads to hang/server crash.
While [un]installing audit_log plugin
a thread acquires the LOCK_plugin mutex
and after successful initialization tries
to write in mysql.plugin table. It holds
this mutex for a long time. If some how
plugin table is corrupted then a write to
plugin table will throw an error, thread try
to log this error in the audit_log plugin,
doing so it tries to acquire the mutex
again and results is server hang/crash.
SOLUTION: Releasing the LOCK_plugin mutex before
writing in mysql.plugin table. We dont
need to hold this mutex as thread already
acquired a TL_WRITE lock on mysql.plugin
table.
ENABLE AUDI PLUGIN WHEN DDL
OPERATION HAPPENING
PROBLEM: While unloading the plugin, state is
not checked before it is to be reaped.
This can lead to simultaneous free of
plugin memory by more than one thread.
Multiple deallocation leads to server
crash. In the present bug two threads
deallocate the alog_log plugin.
SOLUTION: A check is added to ensure that only
one thread is unloading the plugin.
NOTE: No mtr test is added as it requires
multiple threads to access critical
section. debug_sync cannot be used in
the current senario because we dont
have access to thread pointer in
some of the plugin functions. IMHO no
test case in the current time frame.
NUMBERS
If a system variable was declared as deprecated without mention of an
alternative, the message would look funny, e.g. for @@delayed_insert_limit:
Warning 1287 '@@delayed_insert_limit' is deprecated and
will be removed in MySQL .
The message was meant to display the version number, but it's not
possible to give one when declaring a system variable.
The fix does two things:
1) The definition of the message
ER_WARN_DEPRECATED_SYNTAX_NO_REPLACEMENT is changed so that it does
not display a version number. I.e. in English the message now reads:
Warning 1287 The syntax '@@delayed_insert_limit' is deprecated and
will be removed in a future version.
2) The message ER_WARN_DEPRECATED_SYNTAX_WITH_VER is discontinued in
favor of ER_WARN_DEPRECATED_SYNTAX for system variables. This change
was already done in versions 5.6 and above as part of wl#5265. This
part is simply back-ported from the worklog.
and cryptic error 1126 message
The problem was that dlopen() related code was using just a subset
of the path normalization routines used in other places.
Fixed the expansion of the pre-dlopen() behavior for plugins and UDFs
to use a platform-dependent consistent encoding of the paths.
Fixed the error dlopen() error handling to take the correct error message
and strip off the trailing newline character(s).
Fixed tests to do a platform independent replace of directories and to
account for the traling slash.
ALTER TABLE AND/OR PLUGIN/SEMISYNC
If a plugin was uninstalled, thread local values for plugin
variables of string type with PLUGIN_VAR_MEMALLOC flag were
not freed.
With this patch these variables are freed when thread is
done (like all other variables).
sql/sql_class.h:
Added variable which stores memory hunks allocated
for PLUGIN_VAR_MEMALLOC values.
sql/sql_plugin.cc:
Normally all memory allocated for dynamic variables values must
be freed by cleanup_variables().
But if a plugin was uninstalled, descriptors of it's system
variables are lost. Still some memory may be occupied for thread
local values. It is ok for most kinds of variables, as they're
stored on dynamic_variables_ptr and freed when thread is done.
Values for PLUGIN_VAR_MEMALLOC variables are stored separately.
These lost values are handled by plugin_var_memalloc_free().
In sql_class.cc, 'row_count', of type 'ha_rows', was used as last argument for
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD which is
"Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld".
So 'ha_rows' was used as 'long'.
On SPARC32 Solaris builds, 'long' is 4 bytes and 'ha_rows' is 'longlong' i.e. 8 bytes.
So the printf-like code was reading only the first 4 bytes.
Because the CPU is big-endian, 1LL is 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01
so the first four bytes yield 0. So the warning message had "row 0" instead of
"row 1" in test outfile_loaddata.test:
-Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1
+Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 0
All error-messaging functions which internally invoke some printf-life function
are potential candidate for such mistakes.
One apparently easy way to catch such mistakes is to use
ATTRIBUTE_FORMAT (from my_attribute.h).
But this works only when call site has both:
a) the format as a string literal
b) the types of arguments.
So:
func(ER(ER_BLAH), 10);
will silently not be checked, because ER(ER_BLAH) is not known at
compile time (it is known at run-time, and depends on the chosen
language).
And
func("%s", a va_list argument);
has the same problem, as the *real* type of arguments is not
known at this site at compile time (it's known in some caller).
Moreover,
func(ER(ER_BLAH));
though possibly correct (if ER(ER_BLAH) has no '%' markers), will not
compile (gcc says "error: format not a string literal and no format
arguments").
Consequences:
1) ATTRIBUTE_FORMAT is here added only to functions which in practice
take "string literal" formats: "my_error_reporter" and "print_admin_msg".
2) it cannot be added to the other functions: my_error(),
push_warning_printf(), Table_check_intact::report_error(),
general_log_print().
To do a one-time check of functions listed in (2), the following
"static code analysis" has been done:
1) replace
my_error(ER_xxx, arguments for substitution in format)
with the equivalent
my_printf_error(ER_xxx,ER(ER_xxx), arguments for substitution in
format),
so that we have ER(ER_xxx) and the arguments *in the same call site*
2) add ATTRIBUTE_FORMAT to push_warning_printf(),
Table_check_intact::report_error(), general_log_print()
3) replace ER(xxx) with the hard-coded English text found in
errmsg.txt (like: ER(ER_UNKNOWN_ERROR) is replaced with
"Unknown error"), so that a call site has the format as string literal
4) this way, ATTRIBUTE_FORMAT can effectively do its job
5) compile, fix errors detected by ATTRIBUTE_FORMAT
6) revert steps 1-2-3.
The present patch has no compiler error when submitted again to the
static code analysis above.
It cannot catch all problems though: see Field::set_warning(), in
which a call to push_warning_printf() has a variable error
(thus, not replacable by a string literal); I checked set_warning() calls
by hand though.
See also WL 5883 for one proposal to avoid such bugs from appearing
again in the future.
The issues fixed in the patch are:
a) mismatch in types (like 'int' passed to '%ld')
b) more arguments passed than specified in the format.
This patch resolves mismatches by changing the type/number of arguments,
not by changing error messages of sql/share/errmsg.txt. The latter would be wrong,
per the following old rule: errmsg.txt must be as stable as possible; no insertions
or deletions of messages, no changes of type or number of printf-like format specifiers,
are allowed, as long as the change impacts a message already released in a GA version.
If this rule is not followed:
- Connectors, which use error message numbers, will be confused (by insertions/deletions
of messages)
- using errmsg.sys of MySQL 5.1.n with mysqld of MySQL 5.1.(n+1)
could produce wrong messages or crash; such usage can easily happen if
installing 5.1.(n+1) while /etc/my.cnf still has --language=/path/to/5.1.n/xxx;
or if copying mysqld from 5.1.(n+1) into a 5.1.n installation.
When fixing b), I have verified that the superfluous arguments were not used in the format
in the first 5.1 GA (5.1.30 'bteam@astra04-20081114162938-z8mctjp6st27uobm').
Had they been used, then passing them today, even if the message doesn't use them
anymore, would have been necessary, as explained above.
include/my_getopt.h:
this function pointer is used only with "string literal" formats, so we can add
ATTRIBUTE_FORMAT.
mysql-test/collections/default.experimental:
test should pass now
sql/derror.cc:
by having a format as string literal, ATTRIBUTE_FORMAT check becomes effective.
sql/events.cc:
Change justified by the following excerpt from sql/share/errmsg.txt:
ER_EVENT_SAME_NAME
eng "Same old and new event name"
ER_EVENT_SET_VAR_ERROR
eng "Error during starting/stopping of the scheduler. Error code %u"
sql/field.cc:
ER_TOO_BIG_SCALE 42000 S1009
eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu."
ER_TOO_BIG_PRECISION 42000 S1009
eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu."
ER_TOO_BIG_DISPLAYWIDTH 42000 S1009
eng "Display width out of range for column '%-.192s' (max = %lu)"
sql/ha_ndbcluster.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
(sizeof() returns size_t)
sql/ha_ndbcluster_binlog.cc:
Too many arguments for:
ER_GET_ERRMSG
eng "Got error %d '%-.100s' from %s"
Patch by Jonas Oreland.
sql/ha_partition.cc:
print_admin_msg() is used only with a literal as format, so ATTRIBUTE_FORMAT
works.
sql/handler.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
(sizeof() returns size_t)
sql/item_create.cc:
ER_TOO_BIG_SCALE 42000 S1009
eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu."
ER_TOO_BIG_PRECISION 42000 S1009
eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu."
'c_len' and 'c_dec' are char*, passed as %d !! We don't know their value
(as strtoul() failed), but they are likely big, so we use INT_MAX.
'len' is ulong.
sql/item_func.cc:
ER_WARN_DATA_OUT_OF_RANGE 22003
eng "Out of range value for column '%s' at row %ld"
ER_CANT_FIND_UDF
eng "Can't load function '%-.192s'"
sql/item_strfunc.cc:
ER_TOO_BIG_FOR_UNCOMPRESS
eng "Uncompressed data size too large; the maximum size is %d (probably, length of uncompressed data was corrupted)"
max_allowed_packet is ulong.
sql/mysql_priv.h:
sql_print_message_func is a function _pointer_.
sql/sp_head.cc:
ER_SP_RECURSION_LIMIT
eng "Recursive limit %d (as set by the max_sp_recursion_depth variable) was exceeded for routine %.192s"
max_sp_recursion_depth is ulong
sql/sql_acl.cc:
ER_PASSWORD_NO_MATCH 42000
eng "Can't find any matching row in the user table"
ER_CANT_CREATE_USER_WITH_GRANT 42000
eng "You are not allowed to create a user with GRANT"
sql/sql_base.cc:
ER_NOT_KEYFILE
eng "Incorrect key file for table '%-.200s'; try to repair it"
ER_TOO_MANY_TABLES
eng "Too many tables; MySQL can only use %d tables in a join"
MAX_TABLES is size_t.
sql/sql_binlog.cc:
ER_UNKNOWN_ERROR
eng "Unknown error"
sql/sql_class.cc:
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD
eng "Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld"
WARN_DATA_TRUNCATED 01000
eng "Data truncated for column '%s' at row %ld"
sql/sql_connect.cc:
ER_HANDSHAKE_ERROR 08S01
eng "Bad handshake"
ER_BAD_HOST_ERROR 08S01
eng "Can't get hostname for your address"
sql/sql_insert.cc:
ER_WRONG_VALUE_COUNT_ON_ROW 21S01
eng "Column count doesn't match value count at row %ld"
sql/sql_parse.cc:
ER_WARN_HOSTNAME_WONT_WORK
eng "MySQL is started in --skip-name-resolve mode; you must restart it without this switch for this grant to work"
ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT
eng "Too high level of nesting for select"
ER_UNKNOWN_ERROR
eng "Unknown error"
sql/sql_partition.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_plugin.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_prepare.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
ER_UNKNOWN_STMT_HANDLER
eng "Unknown prepared statement handler (%.*s) given to %s"
length value (for '%.*s') must be 'int', per the doc of printf()
and the code of my_vsnprintf().
sql/sql_show.cc:
ER_OUTOFMEMORY HY001 S1001
eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_table.cc:
ER_TOO_BIG_FIELDLENGTH 42000 S1009
eng "Column length too big for column '%-.192s' (max = %lu); use BLOB or TEXT instead"
sql/table.cc:
ER_NOT_FORM_FILE
eng "Incorrect information in file: '%-.200s'"
ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE
eng "Column count of mysql.%s is wrong. Expected %d, found %d. Created with MySQL %d, now running %d. Please use mysql_upgrade to fix this error."
table->s->mysql_version is ulong.
sql/unireg.cc:
ER_TOO_LONG_TABLE_COMMENT
eng "Comment for table '%-.64s' is too long (max = %lu)"
ER_TOO_LONG_FIELD_COMMENT
eng "Comment for field '%-.64s' is too long (max = %lu)"
ER_TOO_BIG_ROWSIZE 42000
eng "Row size too large. The maximum row size for the used table type, not counting BLOBs, is %ld. You have to change some columns to TEXT or BLOBs"
When installing plugins, there is a missing check
for slash (/) in the path on Windows. Note that on
Windows, both / and \ can be used to separate
directories.
This patch fixes the issue by:
- Adding a FN_DIRSEP symbol for all platforms
consisting of a string of legal directory
separators.
- Adding a charset-aware version of strcspn().
- Adding a check_valid_path() function that uses
my_strcspn() to check if any FN_DIRSEP character
is in the supplied string.
- Using the check_valid_path() function in
sql_plugin.cc and sql_udf.cc (which means
replacing the existing test there).
include/config-netware.h:
Adding FN_DIRSEP
******
Adding FN_DIRSEP
include/config-win.h:
Adding FN_DIRSEP
******
Adding FN_DIRSEP
include/m_ctype.h:
Adding my_strspn() and my_strcspn().
******
Adding my_strspn() and my_strcspn().
include/my_global.h:
Adding FN_DIRSEP
******
Adding FN_DIRSEP
mysql-test/t/plugin_not_embedded.test:
Adding test that file names containing / is
disallowed on *all* platforms.
******
Adding test that file names containing / is
disallowed on *all* platforms.
sql/sql_plugin.cc:
Introducing check_if_path() function for
checking if filename is a path to include
/ on Windows.
******
Introducing check_if_path() function for
checking if filename is a path to include
/ on Windows.
sql/sql_udf.cc:
Switching to use check_if_path() function.
******
Switching to use check_if_path() function.
strings/my_strchr.c:
Adding my_strspn() and my_strcspn().
******
Adding my_strspn() and my_strcspn().
If a relative path is supplied to option --defaults-file or
--defaults-extra-file, the server will crash when executing
an INSTALL PLUGIN command. The reason is that the defaults
file is initially read relative the current working directory
when the server is started, but when INSTALL PLUGIN is executed,
the server has changed working directory to the data directory.
Since there is no check that the call to my_load_defaults()
inside mysql_install_plugin(), the subsequence call to
free_defaults() will crash the server.
This patch fixes the problem by:
- Prepending the current working directory to the file name when
a relative path is given to the --defaults-file or --defaults-
extra-file option the first time my_load_defaults() is called,
which is just after the server has started in main().
- Adding a check of the return value of my_load_defaults() inside
mysql_install_plugin() and aborting command (with an error) if
an error is returned.
- It also adds a check of the return value for load_defaults in
lib_sql.cc for the embedded server since that was missing.
To test that the relative files for the options --defaults-file and
--defaults-extra-file is handled properly, mysql-test-run.pl is also
changed to not add a --defaults-file option if one is provided in the
tests *.opt file.
This patch implements "permanent" load option for
plugins as specified by WL#5341.
mysql-test/r/plugin_load_option.result:
A test case for WL#5341.
mysql-test/t/plugin_load_option-master.opt:
A test case for WL#5341.
mysql-test/t/plugin_load_option.test:
A test case for WL#5341.
sql/share/errmsg-utf8.txt:
An error message for WL#5341.
sql/sql_plugin.cc:
Added FORCE_PLUS_PERMANENT plugin load option.
sql/sql_plugin.h:
Expose and use plugin load option instead of
is_mandatory flag. This is a requirement for
to-be-implemented WL5496.
attempt to install a plugin twice
Server crashes when [UN]INSTALL PLUGIN fails (returns an
error) and general log is disabled and there are audit
plugins interested in MYSQL_AUDIT_GENERAL_CLASS.
When audit event is triggered, audit subsystem acquires interested
plugins by walking through plugin list. Evidently plugin list
iterator protects plugin list by acquiring LOCK_plugin, see
plugin_foreach_with_mask().
On the other hand [UN]INSTALL PLUGIN is acquiring LOCK_plugin
rather for a long time.
When audit event is triggered during [UN]INSTALL PLUGIN, plugin
list iterator acquires the same lock (within the same thread)
second time.
Repeatable only with general_log disabled, because general_log
triggers MYSQL_AUDIT_GENERAL_LOG event, which acquires audit
plugins before [UN]INSTALL PLUGIN acquired LOCK_plugin.
With this fix we pre-acquire audit plugins for events that
may potentially occur during [UN]INSTALL PLUGIN.
This hack should be removed when LOCK_plugin is fixed so it
protects only what it supposed to protect.
No test case for this fix - we do not have facility to test
audit plugins yet.
sql/sql_audit.cc:
Move "acquire audit plugin" logics to a separate
function.
sql/sql_audit.h:
Move "acquire audit plugin" logics to a separate
function.
sql/sql_plugin.cc:
Pre-acquire audit plugins for events that may potentially occur
during [UN]INSTALL PLUGIN.
check_if_table_exists()
This assert was triggered when the server tried to load plugins
while running in embedded server mode. In embedded server mode,
check_if_table_exists() was used to check if mysql.plugin existed
so that ER_NO_SUCH_TABLE could be silently ignored.
The problem was that this check was done without acquiring a metadata
lock on mysql.plugin first. This triggered the assert.
This patch fixes the problem by removing the call to
check_if_table_exists() from plugin_load(). Instead an error handler
which traps ER_NO_SUCH_TABLE is installed before trying to open
mysql.plugin when running in embedded server mode.
No test coverage added since this assert was triggered by
existing tests running in embedded server mode.
sql/sql_base.cc:
Renamed Prelock_error_handler to No_such_table_error_handler
and moved the declaration to sql_base.h to make it usable
in plugin_load().
sql/sql_base.h:
Renamed Prelock_error_handler to No_such_table_error_handler
and moved the declaration to sql_base.h to make it usable
in plugin_load().
sql/sql_plugin.cc:
Removed call to check_if_table_exists() used to check for mysql.plugin
in plugin_load() for embedded server. Instead install error handler
which traps ER_NO_SUCH_TABLE during open_and_lock_tables().
Reverted the ulong->uint diff
Re-applied the first diff.
The original commit message follows:
enum plugin system variables are ulong internally, not int.
On systems where long is not the same as an int it causes
problems.
Fixed by correct typecasting. Removed the test from the
experimental list.
The enum system variables were handled inconsistently
as ints, unsigned int and unsigned long on various places.
This caused problems on platforms on which
sizeof(int) != sizeof(long).
Fixed by homogenizing the type of the enum variables
to unsigned int, since it's size compatible with the C enum
type.
Removed the test from the experimental list.
This patch also fixes Bug#55452 "SET PASSWORD is
replicated twice in RBR mode".
The goal of this patch is to remove the release of
metadata locks from close_thread_tables().
This is necessary to not mistakenly release
the locks in the course of a multi-step
operation that involves multiple close_thread_tables()
or close_tables_for_reopen().
On the same token, move statement commit outside
close_thread_tables().
Other cleanups:
Cleanup COM_FIELD_LIST.
Don't call close_thread_tables() in COM_SHUTDOWN -- there
are no open tables there that can be closed (we leave
the locked tables mode in THD destructor, and this
close_thread_tables() won't leave it anyway).
Make open_and_lock_tables() and open_and_lock_tables_derived()
call close_thread_tables() upon failure.
Remove the calls to close_thread_tables() that are now
unnecessary.
Simplify the back off condition in Open_table_context.
Streamline metadata lock handling in LOCK TABLES
implementation.
Add asserts to ensure correct life cycle of
statement transaction in a session.
Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.
mysql-test/r/variables.result:
Update results: set @@autocommit and statement transaction/
prelocked mode.
mysql-test/r/view.result:
A harmless change in CHECK TABLE <view> status for a broken view.
If previously a failure to prelock all functions used in a view
would leave the connection in LTM_PRELOCKED mode, now we call
close_thread_tables() from open_and_lock_tables()
and leave prelocked mode, thus some check in mysql_admin_table() that
works only in prelocked/locked tables mode is no longer activated.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
Fixed Bug#55452 "SET PASSWORD is replicated twice in
RBR mode": extra binlog events are gone from the
binary log.
mysql-test/t/variables.test:
Add a test case: set autocommit and statement transaction/prelocked
mode.
sql/event_data_objects.cc:
Simplify code in Event_job_data::execute().
Move sp_head memory management to lex_end().
sql/event_db_repository.cc:
Move the release of metadata locks outside
close_thread_tables().
Make sure we call close_thread_tables() when
open_and_lock_tables() fails and remove extra
code from the events data dictionary.
Use close_mysql_tables(), a new internal
function to properly close mysql.* tables
in the data dictionary.
Contract Event_db_repository::drop_events_by_field,
drop_schema_events into one function.
When dropping all events in a schema,
make sure we don't mistakenly release all
locks acquired by DROP DATABASE. These
include locks on the database name
and the global intention exclusive
metadata lock.
sql/event_db_repository.h:
Function open_event_table() does not require an instance
of Event_db_repository.
sql/events.cc:
Use close_mysql_tables() instead of close_thread_tables()
to bootstrap events, since the latter no longer
releases metadata locks.
sql/ha_ndbcluster.cc:
- mysql_rm_table_part2 no longer releases
acquired metadata locks. Do it in the caller.
sql/ha_ndbcluster_binlog.cc:
Deploy the new protocol for closing thread
tables in run_query() and ndb_binlog_index
code.
sql/handler.cc:
Assert that we never call ha_commit_trans/
ha_rollback_trans in sub-statement, which
is now the case.
sql/handler.h:
Add an accessor to check whether THD_TRANS object
is empty (has no transaction started).
sql/log.cc:
Update a comment.
sql/log_event.cc:
Since now we commit/rollback statement transaction in
mysql_execute_command(), we need a mechanism to communicate
from Query_log_event::do_apply_event() to mysql_execute_command()
that the statement transaction should be rolled back, not committed.
Ideally it would be a virtual method of THD. I hesitate
to make THD a virtual base class in this already large patch.
Use a thd->variables.option_bits for now.
Remove a call to close_thread_tables() from the slave IO
thread. It doesn't open any tables, and the protocol
for closing thread tables is more complicated now.
Make sure we properly close thread tables, however,
in Load_data_log_event, which doesn't
follow the standard server execution procedure
with mysql_execute_command().
@todo: this piece should use Server_runnable
framework instead.
Remove an unnecessary call to mysql_unlock_tables().
sql/rpl_rli.cc:
Update Relay_log_info::slave_close_thread_tables()
to follow the new close protocol.
sql/set_var.cc:
Remove an unused header.
sql/slave.cc:
Remove an unnecessary call to
close_thread_tables().
sql/sp.cc:
Remove unnecessary calls to close_thread_tables()
from SP DDL implementation. The tables will
be closed by the caller, in mysql_execute_command().
When dropping all routines in a database, make sure
to not mistakenly drop all metadata locks acquired
so far, they include the scoped lock on the schema.
sql/sp_head.cc:
Correct the protocol that closes thread tables
in an SP instruction.
Clear lex->sphead before cleaning up lex
with lex_end to make sure that we don't
delete the sphead twice. It's considered
to be "cleaner" and more in line with
future changes than calling delete lex->sphead
in other places that cleanup the lex.
sql/sp_head.h:
When destroying m_lex_keeper of an instruction,
don't delete the sphead that all lex objects
share.
@todo: don't store a reference to routine's sp_head
instance in instruction's lex.
sql/sql_acl.cc:
Don't call close_thread_tables() where the caller will
do that for us.
Fix Bug#55452 "SET PASSWORD is replicated twice in RBR
mode" by disabling RBR replication in change_password()
function.
Use close_mysql_tables() in bootstrap and ACL reload
code to make sure we release all metadata locks.
sql/sql_base.cc:
This is the main part of the patch:
- remove manipulation with thd->transaction
and thd->mdl_context from close_thread_tables().
Now this function is only responsible for closing
tables, nothing else.
This is necessary to be able to easily use
close_thread_tables() in procedures, that
involve multiple open/close tables, which all
need to be protected continuously by metadata
locks.
Add asserts ensuring that TABLE object
is only used when is protected by a metadata lock.
Simplify the back off condition of Open_table_context,
we no longer need to look at the autocommit mode.
Make open_and_lock_tables() and open_normal_and_derived_tables()
close thread tables and release metadata locks acquired so-far
upon failure. This simplifies their usage.
Implement close_mysql_tables().
sql/sql_base.h:
Add declaration for close_mysql_tables().
sql/sql_class.cc:
Remove a piece of dead code that has also become redundant
after the fix for Bug 37521.
The code became dead when my_eof() was made a non-protocol method,
but a method that merely modifies the diagnostics area.
The code became redundant with the fix for Bug#37521, when
we started to cal close_thread_tables() before
Protocol::end_statement().
sql/sql_do.cc:
Do nothing in DO if inside a substatement
(the assert moved out of trans_rollback_stmt).
sql/sql_handler.cc:
Add comments.
sql/sql_insert.cc:
Remove dead code.
Release metadata locks explicitly at the
end of the delayed insert thread.
sql/sql_lex.cc:
Add destruction of lex->sphead to lex_end(),
lex "reset" method called at the end of each statement.
sql/sql_parse.cc:
Move close_thread_tables() and other related
cleanups to mysql_execute_command()
from dispatch_command(). This has become
possible after the fix for Bug#37521.
Mark federated SERVER statements as DDL.
Next step: make sure that we don't store
eof packet in the query cache, and move
the query cache code outside mysql_parse.
Brush up the code of COM_FIELD_LIST.
Remove unnecessary calls to close_thread_tables().
When killing a query, don't report "OK"
if it was a suicide.
sql/sql_parse.h:
Remove declaration of a function that is now static.
sql/sql_partition.cc:
Remove an unnecessary call to close_thread_tables().
sql/sql_plugin.cc:
open_and_lock_tables() will clean up
after itself after a failure.
Move close_thread_tables() above
end: label, and replace with close_mysql_tables(),
which will also release the metadata lock
on mysql.plugin.
sql/sql_prepare.cc:
Now that we no longer release locks in close_thread_tables()
statement prepare code has become more straightforward.
Remove the now redundant check for thd->killed() (used
only by the backup project) from Execute_server_runnable.
Reorder code to take into account that now mysql_execute_command()
performs lex->unit.cleanup() and close_thread_tables().
sql/sql_priv.h:
Add a new option to server options to interact
between the slave SQL thread and execution
framework (hack). @todo: use a virtual
method of class THD instead.
sql/sql_servers.cc:
Due to Bug 25705 replication of
DROP/CREATE/ALTER SERVER is broken.
Make sure at least we do not attempt to
replicate these statements using RBR,
as this violates the assert in close_mysql_tables().
sql/sql_table.cc:
Do not release metadata locks in mysql_rm_table_part2,
this is done by the caller.
Do not call close_thread_tables() in mysql_create_table(),
this is done by the caller.
Fix a bug in DROP TABLE under LOCK TABLES when,
upon error in wait_while_table_is_used() we would mistakenly
release the metadata lock on a non-dropped table.
Explicitly release metadata locks when doing an implicit
commit.
sql/sql_trigger.cc:
Now that we delete lex->sphead in lex_end(),
zero the trigger's sphead in lex after loading
the trigger, to avoid double deletion.
sql/sql_udf.cc:
Use close_mysql_tables() instead of close_thread_tables().
sql/sys_vars.cc:
Remove code added in scope of WL#4284 which would
break when we perform set @@session.autocommit along
with setting other variables and using tables or functions.
A test case added to variables.test.
sql/transaction.cc:
Add asserts.
sql/tztime.cc:
Use close_mysql_tables() rather than close_thread_tables().
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
an atomic counter"
Split the large LOCK_open section in open_table().
Do not call open_table_from_share() under LOCK_open.
Remove thd->version.
This fixes
Bug#50589 "Server hang on a query evaluated using a temporary
table"
Bug#51557 "LOCK_open and kernel_mutex are not happy together"
Bug#49463 "LOCK_table and innodb are not nice when handler
instances are created".
This patch has effect on storage engines that rely on
ha_open() PSEA method being called under LOCK_open.
In particular:
1) NDB is broken and left unfixed. NDB relies on LOCK_open
being kept as part of ha_open(), since it uses auto-discovery.
While previously the NDB open code was race-prone, now
it simply fails on asserts.
2) HEAP engine had a race in ha_heap::open() when
a share for the same table could be added twice
to the list of shares, or a dangling reference to a share
stored in HEAP handler. This patch aims to address this
problem by 'pinning' the newly created share in the
internal HEAP engine share list until at least one
handler instance is created using that share.
include/heap.h:
Add members to HP_CREATE_INFO.
Declare heap_release_share().
sql/lock.cc:
Remove thd->version, use thd->open_tables->s->version instead.
sql/repl_failsafe.cc:
Remove thd->version.
sql/sql_base.cc:
- close_thread_table(): move handler cleanup code outside the critical section protected by LOCK_open.
- remove thd->version
- split the large critical section in open_table() that
opens a new table from share and is protected by LOCK_open
into 2 critical sections, thus reducing the critical path.
- make check_if_table_exists() acquire LOCK_open internally.
- use thd->open_tables->s->version instead of thd->refresh_version to make sure that all tables in
thd->open_tables are in the same refresh series.
sql/sql_base.h:
Add declaration for check_if_table_exists().
sql/sql_class.cc:
Remove init_open_tables_state(), it's now equal to
reset_open_tables_state().
sql/sql_class.h:
Remove thd->version, THD::init_open_tables_state().
sql/sql_plugin.cc:
Use table->m_needs_reopen to mark the table as stale
rather than manipulate with thd->version, which is no more.
sql/sql_udf.cc:
Use table->m_needs_reopen to mark the table as stale
rather than manipulate with thd->version, which is no more.
sql/table.h:
Remove an unused variable.
sql/tztime.cc:
Use table->m_needs_reopen to mark the table as stale
rather than manipulate with thd->version, which is no more.
storage/heap/CMakeLists.txt:
Add heap tests to cmake build files.
storage/heap/ha_heap.cc:
Fix a race when ha_heap::ha_open() could insert two
HP_SHARE objects into the internal share list or store
a dangling reference to a share in ha_heap instance,
or wrongly set implicit_emptied.
storage/heap/hp_create.c:
Optionally pin a newly created share in the list of shares
by increasing its open_count. This is necessary to
make sure that a newly created share doesn't disappear while
a HP_INFO object is being created to reference it.
storage/heap/hp_open.c:
When adding a new HP_INFO object to the list of objects
in the heap share, make sure the open_count is not increased
twice.
storage/heap/hp_test1.c:
Adjust the test to new function signatures.
storage/heap/hp_test2.c:
Adjust the test to new function signatures.
strict aliasing violations.
Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.
As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.
The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.
client/mysql.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysql_upgrade.c:
Remove now-unnecessary casts.
client/mysqladmin.cc:
Remove now-unnecessary casts.
Break up large strings.
client/mysqlbinlog.cc:
Remove now-unnecessary casts.
client/mysqlcheck.c:
Remove now-unnecessary casts.
client/mysqldump.c:
Remove now-unnecessary casts.
client/mysqlimport.c:
Remove now-unnecessary casts.
client/mysqlshow.c:
Remove now-unnecessary casts.
client/mysqlslap.c:
Remove now-unnecessary casts.
client/mysqltest.cc:
Remove now-unnecessary casts.
extra/comp_err.c:
Remove now-unnecessary casts.
extra/my_print_defaults.c:
Remove now-unnecessary casts.
Break up large strings.
extra/mysql_waitpid.c:
Remove now-unnecessary casts.
extra/perror.c:
Remove now-unnecessary casts.
extra/resolve_stack_dump.c:
Remove now-unnecessary casts.
extra/resolveip.c:
Remove now-unnecessary casts.
include/my_getopt.h:
Use a void pointer type as the opaque type to avoid problems with type
incompatibility -- GCC issues warnings when the type name is not type
compatible with a operand. As a side bonus, a explicit cast won't be
necessary anymore.
include/sslopt-longopts.h:
Remove now-unnecessary casts.
Break up large strings.
mysys/my_getopt.c:
Update opaque type and introduce a type definition for the
argument to my_getopt_register_get_addr.
server-tools/instance-manager/options.cc:
Remove now-unnecessary casts.
sql/mysqld.cc:
Remove now-unnecessary casts.
Break up large strings.
Update mysql_getopt_value prototype (the old prototype
was different from the definition anyway).
sql/sql_plugin.cc:
The type of a pointer to a function must be compatible with the
pointed-to function type, otherwise the behavior is undefined.
sql/table.cc:
The variable buf pointer to pointer to pointer to constant char
could improperly alias a incompatible type in call to fix_type_
pointers. Since this was actually dead code, it is simply removed.
sql/unireg.cc:
Remove call to get_form_pos. The code creates a new FRM file which
is always truncated and writes the form position as 0. Hence, no
need to retrieve it, we now for sure it is 0.
storage/archive/archive_reader.c:
Remove now-unnecessary casts.
storage/myisam/ft_nlq_search.c:
Read weight directly from the buffer.
storage/myisam/fulltext.h:
Add explanation about the type duality of a key buffer.
Add accessor macro to retrieve a FT float value.
storage/myisam/mi_test1.c:
Remove now-unnecessary casts.
storage/myisam/myisam_ftdump.c:
Read weight directly from the buffer.
storage/myisam/myisamchk.c:
Remove now-unnecessary casts.
storage/myisam/myisamlog.c:
A pointer to char was used to alias a pointer to pointer to
unsigned char, thus violating strict aliasing rules.
storage/myisam/myisampack.c:
Remove now-unnecessary casts.
strings/decimal.c:
Remove aliasing violation, printing the value is enough for
debugging purposes.
tests/mysql_client_test.c:
Remove now-unnecessary casts.
Conflicts:
Text conflict in configure.in
Text conflict in dbug/dbug.c
Text conflict in mysql-test/r/ps.result
Text conflict in mysql-test/t/ps.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/ha_ndbcluster.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_table.cc
Previously installed dynamic plugins are explicitly not loaded
on startup with --skip-grant-tables enabled. However, INSTALL
PLUGIN/UNINSTALL PLUGIN commands are allowed, and result in
inconsistent error messages (reporting duplicate plugin or
plugin does not exist).
This patch adds a check for --skip-grant-tables mode, and
returns error ER_OPTION_PREVENTS_STATEMENT to the user when
the above commands are attempted.
Potential deadlock situation involving LOCK_plugin,
LOCK_global_system_variables and LOCK_status.
This patch backports the fix from next-mr, unlocking
LOCK_plugin before calling plugin->init() and
add_status_vars().
This patch:
- Moves all definitions from the mysql_priv.h file into
header files for the component where the variable is
defined
- Creates header files if the component lacks one
- Eliminates all include directives from mysql_priv.h
- Eliminates all circular include cycles
- Rename time.cc to sql_time.cc
- Rename mysql_priv.h to sql_priv.h
Conflicts:
Text conflict in client/mysqlbinlog.cc
Text conflict in mysql-test/Makefile.am
Text conflict in mysql-test/collections/default.daily
Text conflict in mysql-test/r/mysqlbinlog_row_innodb.result
Text conflict in mysql-test/suite/rpl/r/rpl_typeconv_innodb.result
Text conflict in mysql-test/suite/rpl/t/rpl_get_master_version_and_clock.test
Text conflict in mysql-test/suite/rpl/t/rpl_row_create_table.test
Text conflict in mysql-test/suite/rpl/t/rpl_slave_skip.test
Text conflict in mysql-test/suite/rpl/t/rpl_typeconv_innodb.test
Text conflict in mysys/charset.c
Text conflict in sql/field.cc
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_func.cc
Text conflict in sql/log.cc
Text conflict in sql/log_event.cc
Text conflict in sql/log_event_old.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/rpl_utility.cc
Text conflict in sql/rpl_utility.h
Text conflict in sql/set_var.cc
Text conflict in sql/share/Makefile.am
Text conflict in sql/sql_delete.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_table.cc
Text conflict in storage/example/ha_example.h
Text conflict in storage/federated/ha_federated.cc
Text conflict in storage/myisammrg/ha_myisammrg.cc
Text conflict in storage/myisammrg/myrg_open.c
Conflicts:
Text conflict in mysql-test/r/partition_innodb.result
Text conflict in sql/field.h
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/item_sum.h
Text conflict in sql/log_event_old.cc
Text conflict in sql/protocol.cc
Text conflict in sql/sql_select.cc
Text conflict in sql/sql_yacc.yy
The problem was that UNINSTALL PLUGIN wasn't performing privilege
checks before removing a plugin. Any user (including users without
any kind of privileges) could uninstall any plugin.
The solution is to verify if the user has the DELETE privilege for
the mysql.plugin table before uninstalling a plugin.
mysql-test/r/plugin_not_embedded.result:
Add test case result for Bug#51770.
mysql-test/t/plugin_not_embedded-master.opt:
Add example plugin path.
mysql-test/t/plugin_not_embedded.test:
Add test case for Bug#51770.
Skip embedded as test relies on privileges checks.
This patch prevents system threads and system table accesses from
using user-specified values for "lock_wait_timeout". Instead all
such accesses are done using the default value (1 year).
This prevents background tasks (such as replication, events,
accessing stored function definitions, logging, reading time-zone
information, etc.) from failing in cases where the global value
of "lock_wait_timeout" is set very low.
The patch also simplifies the open tables API. Rather than adding
another convenience function for opening and locking system tables,
this patch removes most of the existing convenience functions for
open_and_lock_tables_derived(). Before, open_and_lock_tables() was
a convenience function that enforced derived tables handling, while
open_and_lock_tables_derived() was the main function where derived
tables handling was optional. Now, this convencience function is
gone and the main function is renamed to open_and_lock_tables().
No test case added as it would have required the use of --sleep to
check that system threads and system tables have a different timeout
value from the user-specified "lock_wait_timeout" system variable.
mysql-test/t/disabled.def:
Restore disabled ssl tests: SSL certificates were updated.
Disable sp_sync.test, the test case can't work in next-4284.
mysql-test/t/partition_innodb.test:
Disable parsing of the test case for Bug#47343,
the test can not work in next-4284.
mysql-test/t/ps_ddl.test:
Update results (CREATE TABLE IF NOT EXISTS takes
into account existence of the temporary table).
