The problem was that LOAD DATA code (sql_load.cc) didn't take into
account that there may be items, representing references to other
columns. This is a usual case in views. The crash happened because
Item_direct_view_ref was casted to Item_user_var_as_out_param,
which is not a base class.
The fix is to
1) Handle references properly;
2) Ensure that an item is treated as a user variable only when
it is a user variable indeed;
3) Report an error if LOAD DATA is used to load data into
non-updatable column.
Bug #18453 Warning/error message if there is a mismatch between ...
There were three problems:
1. the reported lack of warnings for the BEFORE syntax of PURGE;
2. the similar lack of warnings for the TO syntax;
3. incompatible behaviour between the two in that the latter blanked out
regardlessly of presence or lack the actual file corresponding to
an index record; the former version gave up at the first mismatch.
fixed with deploying the warning's generation and synronizing logics of
purge_logs() and purge_logs_before_date().
my_stat() is called in either of two branches of purge_logs() (responsible
for the TO syntax of PURGE) similarly to how it has behaved in the BEFORE syntax.
If there is no actual binlog file, my_stat returns NULL and my_delete is
not invoked.
A critical error is reported to the user if a file from the index
could not be retrieved info about or deleted with a system error code
different than ENOENT.
The error message due to lack of the default value for an extra field
was not as informative as it should be.
Fixed with improving the scheme of gathering, propagating and reporting
errors in applying rows events.
The scheme is in the following.
Any kind of error of processing of a row event incidents are to be
registered with my_error().
In the end Rows_log_event::do_apply_event() invokes rli->report() with the
message to display consisting of all the errors.
This mimics `show warnings' displaying.
A simple test checks three errors in processing an event.
Two hunks - a user level error and pushing it into the list -
have been devoted to already fixed Bug@31702.
Some open issues relating to this artifact listed on BUG@21842 page and
on WL@3679.
Todo: to synchronize the statement in the tests comments on Update and Delete
events may not stop when an extra field does not have a default with wl@3228 spec.
Problem: it is unsafe to read base64-printed events without first
reading the Format_description_log_event (FD). Currently, mysqlbinlog
cannot print the FD.
As a side effect, another bug has also been fixed: When mysqlbinlog
--start-position=X was specified, no ROLLBACK was printed. I changed
this, so that ROLLBACK is always printed.
This patch does several things:
- Format_description_log_event (FD) now print themselves in base64
format.
- mysqlbinlog is now able to print FD events. It has three modes:
--base64-output=auto Print row events in base64 output, and print
FD event. The FD event is printed even if
it is outside the range specified with
--start-position, because it would not be
safe to read row events otherwise. This is
the default.
--base64-output=always Like --base64-output=auto, but also print
base64 output for query events. This is
like the old --base64-output flag, which
is also a shorthand for
--base64-output=always
--base64-output=never Never print base64 output, generate error if
row events occur in binlog. This is
useful to suppress the FD event in binlogs
known not to contain row events (e.g.,
because BINLOG statement is unsafe,
requires root privileges, is not SQL, etc)
- the BINLOG statement now handles FD events correctly, by setting
the thread's rli's relay log's description_event_for_exec to the
loaded event.
In fact, executing a BINLOG statement is almost the same as reading
an event from a relay log. Before my patch, the code for this was
separated (exec_relay_log_event in slave.cc executes events from
the relay log, mysql_client_binlog_statement in sql_binlog.cc
executes BINLOG statements). I needed to augment
mysql_client_binlog_statement to do parts of what
exec_relay_log_event does. Hence, I did a small refactoring and
moved parts of exec_relay_log_event to a new function, which I
named apply_event_and_update_pos. apply_event_and_update_pos is
called both from exec_relay_log_event and from
mysql_client_binlog_statement.
- When a non-FD event is executed in a BINLOG statement, without
previously executing a FD event in a BINLOG statement, it generates
an error, because that's unsafe. I took a new error code for that:
ER_NO_FORMAT_DESCRIPTION_EVENT_BEFORE_BINLOG_STATEMENTS.
In order to get a decent error message containing the name of the
event, I added the class method char*
Log_event::get_type_str(Log_event_type type), which returns a
string name for the given Log_event_type. This is just like the
existing char* Log_event::get_type_str(), except it is a class
method that takes the log event type as parameter.
I also added PRE_GA_*_ROWS_LOG_EVENT to Log_event::get_type_str(),
so that names of old rows event are properly printed.
- When reading an event, I added a check that the event type is known
by the current Format_description_log_event. Without this, it may
crash on bad input (and I was struck by this several times).
- I patched the following test cases, which all contain BINLOG
statements for row events which must be preceded by BINLOG
statements for FD events:
- rpl_bug31076
While I was here, I fixed some small things in log_event.cc:
- replaced hard-coded 4 by EVENT_TYPE_OFFSET in 3 places
- replaced return by DBUG_VOID_RETURN in one place
- The name of the logfile can be '-' to indicate stdin. Before my
patch, the code just checked if the first character is '-'; now it
does a full strcmp(). Probably, all arguments that begin with a -
are already handled somewhere else as flags, but I still think it
is better that the code reflects what it is supposed to do, with as
little dependencies as possible on other parts of the code. If we
one day implement that all command line arguments after -- are
files (as most unix tools do), then we need this.
I also fixed the following in slave.cc:
- next_event() was declared twice, and queue_event was not static but
should be static (not used outside the file).
without PK
Bug#31609 Not all RBR slave errors reported as errors
bug#32468 delete rows event on a table with foreign key constraint fails
The first two bugs comprise idempotency issues.
First, there was no error code reported under conditions of the bug
description although the slave sql thread halted.
Second, executions were different with and without presence of prim key in
the table.
Third, there was no way to instruct the slave whether to ignore an error
and skip to the following event or to halt.
Fourth, there are handler errors which might happen due to idempotent
applying of binlog but those were not listed among the "idempotent" error
list.
All the named issues are addressed.
Wrt to the 3rd, there is the new global system variable, changeble at run
time, which controls the slave sql thread behaviour.
The new variable allows further extensions to mimic the sql_mode
session/global variable.
To address the 4th, the new bug#32468 had to be fixed as it was staying
in the way.
Problem: there was no standard syntax error when
creating partitions with syntax error in
the partitioning clause.
Solution: added "Syntax error: " to the error message
in the SELECT INTO OUTFILE clause starts with a special
character (one of n, t, r, b, 0, Z or N) and ENCLOSED BY
is empty, every occurrence of this character within a
field value is duplicated.
Duplication has been avoided.
New warning message has been added: "First character of
the FIELDS TERMINATED string is ambiguous; please use
non-optional and non-empty FIELDS ENCLOSED BY".
The problem was that the RETURNS column in the mysql.proc was of
CHAR(64). That was not enough for storing long-named datatypes.
The fix is to change CHAR(64) to LONGBLOB, and to throw warnings
at the time a stored routine is created if some data is truncated
during writing into mysql.proc.
Declaring an all space column name in the SELECT FROM DUAL or in a view
leads to misleading warning message:
"Leading spaces are removed from name ' '".
The Item::set_name method has been modified to raise warnings like
"Name ' ' has become ''" in case of the truncation of an all
space identifier to an empty string identifier instead of the
"Leading spaces are removed from name ' '" warning message.
Based on contributed patch from Martin Friebe, CLA from 2007-02-24.
The parser lacked support for field sizes after signed long,
when it should extend to 2**32-1.
Now, we correct that limitation, and also make the error handling
consistent for casts.
Two character mappings were way off (backtick and tilde were "E"
and "Y"!), and three others were slightly rotated. The first
would cause collisions, and the latter was probably benign.
Now, assign the character mappings exactly to their normal values.
The SELECT query with more than 31 nested dependent SELECT queries returned
wrong result.
New error message has been added: ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT.
It will be reported as: "Too high level of nesting for select".
Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
a deadlock)
Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.
The root cause traces to the following code:
in sql_base.cc, open_table()
if (table->in_use != thd)
{
/* wait_for_condition will unlock LOCK_open for us */
wait_for_condition(thd, &LOCK_open, &COND_refresh);
}
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.
In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY
With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.
See WL#3984 for details related to the new locking design.
Additional changes (misc bugs exposed and fixed):
1)
mysqldump which would ignore some tables in dump_all_tables_in_db(),
but forget to ignore the same in dump_all_views_in_db().
2)
mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.
3)
Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).
4)
Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.
results.
When executing a CREATE EVENT statement with ON COMPLETION NOT PRESERVE
clause (explicit or implicit) and completion date in the past, we do not
create the event. Or, put it differently, we create it and then drop
immediately.
A warning is issued in this case, not an error -- we want to load
successfully old database dumps, and such dumps may contain events
that are no longer valid.
Update the warning text to not imply an erroneous condition.
- BUG#11986: Stored routines and triggers can fail if the code
has a non-ascii symbol
- BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
- BUG#19443: INFORMATION_SCHEMA does not support charsets properly
- BUG#21249: Character set of SP-var can be ignored
- BUG#25212: Character set of string constant is ignored (stored routines)
- BUG#25221: Character set of string constant is ignored (triggers)
There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
definition;
1. No query-definition-character set.
In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.
The context contains the following data:
- client character set;
- connection collation (character set and collation);
- collation of the owner database;
The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).
2. Wrong mysqldump-output.
The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.
Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).
The solution is
- to store definition queries in the original character set;
- to change SHOW CREATE statement to output definition query in the
binary character set (i.e. without any conversion);
- introduce SHOW CREATE TRIGGER statement;
- to dump special statements to switch the context to the original one
before dumping and restore it afterwards.
Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.
3. INFORMATION_SCHEMA showed non-UTF8 strings
The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.
Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.
This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object. Specialized SHOW CREATE statements should be
used for this.
The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).
Example:
- original query:
CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;
- UTF8 query (for INFORMATION_SCHEMA):
CREATE VIEW v1 AS SELECT 'Hello' AS c1;
replication):
Patch to add binlog format capabilities to the InnoDB storage engine.
The engine will not allow statement format logging when in READ COMMITTED
or READ UNCOMMITTED transaction isolation level.
In addition, an error is generated when trying to use READ COMMITTED
or READ UNCOMMITTED transaction isolation level in STATEMENT binlog
mode.
Adding new fields Last_{IO,SQL}_Errno and Last_{IO,SQL}_Error to output
of SHOW SLAVE STATUS to hold errors from I/O and SQL thread respectively.
Old fields Last_Error and Last_Errno are aliases for Last_SQL_Error and
Last_SQL_Errno respectively.
Fields are added last to output of SHOW SLAVE STATUS to allow old applications
to use the same positional arguments into the row, while allowing new
application to benefit from the added information.
In addition, some new error codes are added (especially for the I/O
thread) to be able to provide sensible error message.
SHOW CREATE TABLE fails
Underlying table names, that merge engine fails to open were not
reported.
With this fix CHECK TABLE issued against merge table reports all
underlying table names that it fails to open. Other statements
are unaffected, that is underlying table names are not included
into error message.
This fix doesn't solve SHOW CREATE TABLE issue.
Adding support to allow engines to tell what formats they can handle.
The server will generate an error if it is not possible to log the
statement according to the logging mode in effect.
Adding flags to several storage engines to state what they can handle.
Changes to NDB handler removing code that forces row-based mode and
adding flag saying that NDB can only handle row format.
Adding check that binlog flags are only used for real tables that are
opened for writing.
Replacing binlog_row_based_if_mixed with variable binlog_stmt_flags
holding several flags and adding member functions to manipulate the
flags.
Added code to generate a warning when an attempt to log an unsafe
statement to the binary log was made. The warning is both pushed to the
SHOW WARNINGS table and written to the error log. The prevent flooding
the error log, the warning is just written to the error log once per
open session.
when there are no up-to-date system tables to support it:
- initialize the scheduler before reporting "Ready for connections".
This ensures that warnings, if any, are printed before "Ready for
connections", and this message is not mangled.
- do not abort the scheduler if there are no system tables
- check the tables once at start up, remember the status and disable
the scheduler if the tables are not up to date.
If one attempts to use the scheduler with bad tables,
issue an error message.
- clean up the behaviour of the module under LOCK TABLES and pre-locking
mode
- make sure implicit commit of Events DDL works as expected.
- add more tests
Collateral clean ups in the events code.
This patch fixes Bug#23631 Events: SHOW VARIABLES doesn't work
when mysql.event is damaged
Adding an event that can be used to denote that an incident occured
on the master. The event can be used to denote a gap in the replication
stream, but can also be used to denote other incidents.
In addition, the injector interface is extended with functions to
generate an incident event. The function will also rotate the binary
log after generating an incident event to get a fresh binary log.
BUG#26429: SHOW CREATE EVENT is incorrect for an event that
STARTS NOW()
BUG#26431: Impossible to re-create an event from backup if its
STARTS clause is in the past
WL#3698: Events: execution in local time zone
The problem was that local times specified by the user in AT, STARTS
and ENDS of CREATE EVENT/ALTER EVENT statement were converted to UTC,
and the original time zone was forgotten. This way, event scheduler
couldn't honor Daylight Saving Time shifts, and times shown to the
user were also in UTC. Additionally, CREATE EVENT didn't allow times
in the past, thus preventing straightforward event restoration from
old backups.
This patch reworks event scheduler time computations, performing them
in the time zone associated with the event. Also it allows times to
be in the past.
The patch adds time_zone column to mysql.event table.
NOTE: The patch is almost final, but the bug#9953 should be pushed
first.
The problem was that some facilities (like CONVERT_TZ() function or
server HELP statement) may require implicit access to some tables in
'mysql' database. This access was done by ordinary means of adding
such tables to the list of tables the query is going to open.
However, if we issued LOCK TABLES before that, we would get "table
was not locked" error trying to open such implicit tables.
The solution is to treat certain tables as MySQL system tables, like
we already do for mysql.proc. Such tables may be opened for reading
at any moment regardless of any locks in effect. The cost of this is
that system table may be locked for writing only together with other
system tables, it is disallowed to lock system tables for writing and
have any other lock on any other table.
After this patch the following tables are treated as MySQL system
tables:
mysql.help_category
mysql.help_keyword
mysql.help_relation
mysql.help_topic
mysql.proc (it already was)
mysql.time_zone
mysql.time_zone_leap_second
mysql.time_zone_name
mysql.time_zone_transition
mysql.time_zone_transition_type
These tables are now opened with open_system_tables_for_read() and
closed with close_system_tables(), or one table may be opened with
open_system_table_for_update() and closed with close_thread_tables()
(the latter is used for mysql.proc table, which is updated as part of
normal MySQL server operation). These functions may be used when
some tables were opened and locked already.
NOTE: online update of time zone tables is not possible during
replication, because there's no time zone cache flush neither on LOCK
TABLES, nor on FLUSH TABLES, so the master may serve stale time zone
data from cache, while on slave updated data will be loaded from the
time zone tables.
Before this fix, a call to a User Defined Function (UDF) could,
under some circumstances, be interpreted as a call to a Stored function
instead. This occurred if a native function was invoked in the parameters
for the UDF, as in "select my_udf(abs(x))".
The root cause of this defect is the introduction, by the fix for Bug 21809,
of st_select_lex::udf_list, and it's usage in the parser in sql_yacc.yy
in the rule function_call_generic (in 5.1).
While the fix itself for Bug 21809 is correct in 5.0, the code change
merged into the 5.1 release created the issue, because the calls in 5.1 to :
- lex->current_select->udf_list.push_front(udf)
- lex->current_select->udf_list.pop()
are not balanced in case of native functions, causing the udf_list,
which is really a stack, to be out of sync with the internal stack
maintained by the bison parser.
Instead of moving the call to udf_list.pop(), which would have fixed the
symptom, this patch goes further and removes the need for udf_list.
This is motivated by two reasons:
a) Maintaining a stack in the MySQL code in sync with the stack maintained
internally in sql_yacc.cc (not .yy) is extremely dependent of the
implementation of yacc/bison, and extremely difficult to maintain.
It's also totally dependent of the structure of the grammar, and has a risk
to break with regression defects each time the grammar itself is changed.
b) The previous code did report construct like "foo(expr AS name)" as
syntax errors (ER_PARSER_ERROR), which is incorrect, and misleading.
The syntax is perfectly valid, as this expression is valid when "foo" is
a UDF. Whether this syntax is legal or not depends of the semantic of "foo".
With this change:
a) There is only one stack (in bison), and no List<udf_func> to maintain.
b) "foo(expr AS name)", when used incorrectly, is reported as semantic error:
- ER_WRONG_PARAMETERS_TO_NATIVE_FCT (for native functions)
- ER_WRONG_PARAMETERS_TO_STORED_FCT (for stored functions)
This is achieved by the changes implemented in item_create.cc
Backport of functionality in private 5.2 tree.
Added new language to parser, new mysql.servers table and associated code
to be used by the federated storage engine to allow central connection information
per WL entry.
Bug#21025 (misleading error message when creating functions named 'x', or 'y')
Bug#22619 (Spaces considered harmful)
This change contains a fix to report warnings or errors, and multiple tests
cases.
Before this fix, name collisions between:
- Native functions
- User Defined Functions
- Stored Functions
were not systematically reported, leading to confusing behavior.
I) Native / User Defined Function
Before this fix, is was possible to create a UDF named "foo", with the same
name as a native function "foo", but it was impossible to invoke the UDF,
since the syntax "foo()" always refer to the native function.
After this fix, creating a UDF fails with an error if there is a name
collision with a native function.
II) Native / Stored Function
Before this fix, is was possible to create a SF named "db.foo", with the same
name as a native function "foo", but this was confusing since the syntax
"foo()" would refer to the native function. To refer to the Stored Function,
the user had to use the "db.foo()" syntax.
After this fix, creating a Stored Function reports a warning if there is a
name collision with a native function.
III) User Defined Function / Stored Function
Before this fix, creating a User Defined Function "foo" and a Stored Function
"db.foo" are mutually exclusive operations. Whenever the second function is
created, an error is reported. However, the test suite did not cover this
behavior.
After this fix, the behavior is unchanged, and is now covered by test cases.
Note that the code change in this patch depends on the fix for Bug 21114.
Due to the complexity of this change, everything is documented in WL#3565
This patch is the third iteration, it takes into account the comments
received to date.
Bug #21785 "Server crashes after rename of the log table" and
Bug #21966 "Strange warnings on create like/repair of the log
tables"
According to the patch, from now on, one should use RENAME to
perform a log table rotation (this should also be reflected in
the manual).
Here is a sample:
use mysql;
CREATE TABLE IF NOT EXISTS general_log2 LIKE general_log;
RENAME TABLE general_log TO general_log_backup, general_log2 TO general_log;
The rules for Rename of the log tables are following:
IF 1. Log tables are enabled
AND 2. Rename operates on the log table and nothing is being
renamed to the log table.
DO 3. Throw an error message.
ELSE 4. Perform rename.
The very RENAME query will go the the old (backup) table. This is
consistent with the behavoiur we have with binlog ROTATE LOGS
statement.
Other problems, which are solved by the patch are:
1) Now REPAIR of the log table is exclusive operation (as it should be), this
also eliminates lock-related warnings. and
2) CREATE LIKE TABLE now usese usual read lock on the source table rather
then name lock, which is too restrictive. This way we get rid of another
log table-related warning, which occured because of the above fact
(as a side-effect, name lock resulted in a warning).
