Here is the scenario that causes the failure.(by Mats)
1. The to-be corrupt log event (let's call it X), is split into two
packets B and C on the network level (net_write_buff()). The parts
are X = (x',x''). The part x' ends up in packet B and part x''
ends up in packet C. Prior to the corrupt event X, the event Y has
been written successfully, but has been split into two packets as
well, which we call (y',y'').
2. The master sends packet A = (y'',x') to the slave, increases the
packet sequence number, the slave receives the packet, but fails
to reply before the master gets a timeout.
3. Since the master got a timeout, it reports failure, and aborts
sending the binary log by exiting mysql_binlog_send(). However, it
leaves the buffer intact, still holding y'' (but not x', since the
write_pos is not increased).
4. After exiting mysql_binlog_send(), the master does a
disconnection of the client thread, which involves sending an
error message e to the client (i.e., the slave).
5. In this case, net_write_buff() is used again, but this time the
old contents of the packet is used so that the new packet is
D = (y'',e). Note that this will use a new packet sequence number,
since the packet number was increased in step 2.
6. The slave receives the tail y'' of the Y log event, concatenates
this with x' (which it already received), and writes the event
(x',y'') it to the relay log since it hasn't noticed anything is
amiss.
7. It then tries to read more bytes, which is either e (if the length
given for X just happened to match the length given for Y, or just
plain garbage because the slave is out of sync with what is
actually sent.
8. After a while, the SQL thread tries to execute the event (x',y''),
which is very likely to be just nonsense.
The problem can be fixed by not resetting net->error after the call of
mysql_binlog_send, so the error message will not be sent and the connection
will be closed.
When read_only option was enabled, a user without SUPER privilege could
perform CREATE DATABASE and DROP DATABASE operations.
This patch adds a check to make sure this isn't possible. It also attempts to
simplify the logic used to determine if relevant tables are updated,
making it more human readable.
FLUSH TABLES WITH READ LOCK fails to properly detect write locked
tables when running under low priority updates.
The problem is that when trying to aspire a global read lock, the
reload_acl_and_cache() function fails to properly check if the thread
has a low priority write lock, which later my cause a server crash or
deadlock.
The solution is to simple check if the thread has any type of the
possible exclusive write locks.
The root cause of the issue was that the CREATE FUNCTION grammar,
for User Defined Functions, was using the sp_name rule.
The sp_name rule is intended for fully qualified stored procedure names,
like either ident.ident, or just ident but with a default database
implicitly selected.
A UDF does not have a fully qualified name, only a name (ident), and should
not use the sp_name grammar fragment during parsing.
The fix is to re-organize the CREATE FUNCTION grammar, to better separate:
- creating UDF (no definer, can have AGGREGATE, simple ident)
- creating Stored Functions (definer, no AGGREGATE, fully qualified name)
With the test case provided, another issue was exposed which is also fixed:
the DROP FUNCTION statement was using sp_name and also failing when no database
is implicitly selected, when droping UDF functions.
The fix is also to change the grammar so that DROP FUNCTION works with
both the ident.ident syntax (to drop a stored function), or just the ident
syntax (to drop either a UDF or a Stored Function, in the current database)
similar to bug_27716, but it was stressed on in the synopsis on that there is another
side of the artifact affecting behaviour in transaction.
Fixed with deploying multi_delete::send_error() - otherwise never called - and refining its logic
to perform binlogging job if needed.
The changeset includes the following side effects:
- added tests to check bug_23333's scenarios on the mixture of tables for multi_update;
- fixes bug@30763 with two-liner patch and a test coinciding to one added for bug_23333.
active_mi has been reset (shutdown) at the time of quering with
SHOW SLAVE STATUS so that
at handling of SHOW an attempt to read its members segfaults.
Fixed with checking the value of active_mi before to call show_master_info()
Merely send_ok() is invoked when active_mi does not exist.
A test can not be easiely written.
Notice, there are more analogical cases in the code which require a similar
treatment (to be reported as a bug separately).
The problem was that aborted_threads variable was updated
twice when a client connection had been aborted.
The fix is to refactor a code to have aborted_threads updated
only in one place.
When dumping database from a 4.x server, the mysqldump client
inserted a delimiter sign inside special commentaries of the form:
/*!... CREATE DATABASE IF NOT EXISTS ... ;*/
During restoration that dump file was splitten by delimiter signs on
the client side, and the rest of some commentary strings was prepended
to following statements.
The 4x_server_emul test case option has been added for use with the
DBUG_EXECUTE_IF debugging macro. This option affects debug server
builds only to emulate particular behavior of a 4.x server for
the mysqldump client testing. Non-debugging builds are not affected.
The SELECT query with more than 31 nested dependent SELECT queries returned
wrong result.
New error message has been added: ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT.
It will be reported as: "Too high level of nesting for select".
Problem: thd->thread_specific_used flag is not set executing a statement
containig connection_id() function using PS protocol, that leads to
improper binlog event creation.
Fix: set the flag in the Item_func_connection_id::fix_fields().
Bug #27417 thd->no_trans_update.stmt lost value inside of SF-exec-stack
Once had been set the flag might later got reset inside of a stored routine
execution stack.
The reason was in that there was no check if a new statement started at time
of resetting.
The artifact affects most of binlogable DML queries. Notice, that multi-update
is wrapped up within
bug@27716 fix, multi-delete bug@29136.
Fixed with saving parent's statement flag of whether the statement modified
non-transactional table, and unioning (merging) the value with that was gained
in mysql_execute_command.
Resettling thd->no_trans_update members into thd->transaction.`member`;
Asserting code;
Effectively the following properties are held.
1. At the end of a substatement thd->transaction.stmt.modified_non_trans_table
reflects the fact if such a table got modified by the substatement.
That also respects THD::really_abort_on_warnin() requirements.
2. Eventually thd->transaction.stmt.modified_non_trans_table will be computed as
the union of the values of all invoked sub-statements.
That fixes this bug#27417;
Computing of thd->transaction.all.modified_non_trans_table is refined to base to
the stmt's value for all the case including insert .. select statement which
before the patch had an extra issue bug@28960.
Minor issues are covered with mysql_load, mysql_delete, and binloggin of insert in
to temp_table select.
The supplied test verifies limitely, mostly asserts. The ultimate testing is defered
for bug@13270, bug@23333.
causes full table lock on innodb table.
Also fixes Bug#28502 Triggers that update another innodb table
will block on X lock unnecessarily (duplciate).
Code review fixes.
Both bugs' synopses are misleading: InnoDB table is
not X locked. The statements, however, cannot proceed concurrently,
but this happens due to lock conflicts for tables used in triggers,
not for the InnoDB table.
If a user had an InnoDB table, and two triggers, AFTER UPDATE and
AFTER INSERT, competing for different resources (e.g. two distinct
MyISAM tables), then these two triggers would not be able to execute
concurrently. Moreover, INSERTS/UPDATES of the InnoDB table would
not be able to run concurrently.
The problem had other side-effects (see respective bug reports).
This behavior was a consequence of a shortcoming of the pre-locking
algorithm, which would not distinguish between different DML operations
(e.g. INSERT and DELETE) and pre-lock all the tables
that are used by any trigger defined on the subject table.
The idea of the fix is to extend the pre-locking algorithm to keep track,
for each table, what DML operation it is used for and not
load triggers that are known to never be fired.
Problem: logging queries not using indexes we check a special flag which
is set only at the server startup and is not changing with a corresponding
server variable together.
Fix: check the variable value instead of the flag.
fails if a database is not selected prior.
The problem manifested itself when a user tried to
create a routine that had non-fully-qualified identifiers in its bodies
and there was no current database selected.
This is a regression introduced by the fix for Bug 19022:
The patch for Bug 19022 changes the code to always produce a warning
if we can't resolve the current database in the parser.
In this case this was not necessary, since even though the produced
parsed tree was incorrect, we never re-use sphead
that was obtained at first parsing of CREATE PROCEDURE.
The sphead that is anyhow used is always obtained through db_load_routine,
and there we change the current database to sphead->m_db before
calling yyparse.
The idea of the fix is to resolve the current database directly using
lex->sphead->m_db member when parsing a stored routine body, when
such is present.
This patch removes the need to reset the current database
when loading a trigger or routine definition into SP cache.
The redundant code will be removed in 5.1.
LOCK TABLES takes a list of tables to lock. It may lock several
tables successfully and then encounter a tables that it can't lock,
e.g. because it's locked. In such case it needs to undo the locks on
the already locked tables. And it does that. But it has also notified
the relevant table storage engine handlers that they should lock.
The only reliable way to ensure that the table handlers will give up
their locks is to end the transaction. This is what UNLOCK TABLE
does : it ends the transaction if there were locked tables by LOCK
tables.
It is possible to end the transaction when the lock fails in
LOCK TABLES because LOCK TABLES ends the transaction at its start
already.
Fixed by ending (again) the transaction when LOCK TABLES fails to
lock a table.
ALTER VIEW is currently not supported as a prepared statement
and should be disabled as such as they otherwise could cause server crashes.
ALTER VIEW is currently not supported when called from stored
procedures or functions for related reasons and should also be disabled.
This patch disables these DDL statements and adjusts the appropriate test
cases accordingly.
Additional tests has been added to reflect on the fact that we do support
CREATE/ALTER/DROP TABLE for Prepared Statements (PS), Stored Procedures (SP)
and PS within SP.
protocol
Fixed duplicated code, same as last commit.
One could send a malformed packet that caused the server to SEGV. In
recent versions of the password protocol, the client tells the server
what length the ciphertext is (almost always 20). If that length was
large enough to overflow a signed char, then the number would jump to
very large after being casted to unsigned int.
Instead, cast the *passwd char to uchar.
protocol
One could send a malformed packet that caused the server to SEGV. In
recent versions of the password protocol, the client tells the server
what length the ciphertext is (almost always 20). If that length was
large enough to overflow a signed char, then the number would jump to
very large after being casted to unsigned int.
Instead, cast the *passwd char to uchar.
In case of database level grant the database name may be a pattern,
in case of table|column level grant the database name can not be a pattern.
We use 'dont_check_global_grants' as a flag to determine
if it's database level grant command
(see SQLCOM_GRANT case, mysql_execute_command() function) and
set db_is_pattern according to 'dont_check_global_grants' value.
- The "mysql client in mysqld"(which is used by
replication and federated) should use alarms instead of setting
socket timeout value if the rest of the server uses alarm. By
always calling 'my_net_set_write_timeout'
or 'my_net_set_read_timeout' when changing the timeout value(s), the
selection whether to use alarms or timeouts will be handled by
ifdef's in those two functions.
- Move declaration of 'vio_timeout' into "vio_priv.h"
Bug #23667 "CREATE TABLE LIKE is not isolated from alteration
by other connections"
Bug #18950 "CREATE TABLE LIKE does not obtain LOCK_open"
As well as:
Bug #25578 "CREATE TABLE LIKE does not require any privileges
on source table".
The first and the second bugs resulted in various errors and wrong
binary log order when one tried to execute concurrently CREATE TABLE LIKE
statement and DDL statements on source table or DML/DDL statements on its
target table.
The problem was caused by incomplete protection/table-locking against
concurrent statements implemented in mysql_create_like_table() routine.
We solve it by simply implementing such protection in proper way (see
comment for sql_table.cc for details).
The third bug allowed user who didn't have any privileges on table create
its copy and therefore circumvent privilege check for SHOW CREATE TABLE.
This patch solves this problem by adding privilege check, which was missing.
Finally it also removes some duplicated code from mysql_create_like_table().
Note that, altough tests covering concurrency-related aspects of CREATE TABLE
LIKE behaviour will only be introduced in 5.1, they were run manually for
this patch as well.
Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT
with locked tables"
Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers"
Bug #24738 "CREATE TABLE ... SELECT is not isolated properly"
Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when
temporary table exists"
Deadlock occured when one tried to execute CREATE TABLE IF NOT
EXISTS ... SELECT statement under LOCK TABLES which held
read lock on target table.
Attempt to execute the same statement for already existing
target table with triggers caused server crashes.
Also concurrent execution of CREATE TABLE ... SELECT statement
and other statements involving target table suffered from
various races (some of which might've led to deadlocks).
Finally, attempt to execute CREATE TABLE ... SELECT in case
when a temporary table with same name was already present
led to the insertion of data into this temporary table and
creation of empty non-temporary table.
All above problems stemmed from the old implementation of CREATE
TABLE ... SELECT in which we created, opened and locked target
table without any special protection in a separate step and not
with the rest of tables used by this statement.
This underminded deadlock-avoidance approach used in server
and created window for races. It also excluded target table
from prelocking causing problems with trigger execution.
The patch solves these problems by implementing new approach to
handling of CREATE TABLE ... SELECT for base tables.
We try to open and lock table to be created at the same time as
the rest of tables used by this statement. If such table does not
exist at this moment we create and place in the table cache special
placeholder for it which prevents its creation or any other usage
by other threads.
We still use old approach for creation of temporary tables.
Also note that we decided to postpone introduction of some tests
for concurrent behaviour of CREATE TABLE ... SELECT till 5.1.
The main reason for this is absence in 5.0 ability to set @@debug
variable at runtime, which can be circumvented only by using several
test files with individual .opt files. Since the latter is likely
to slowdown test-suite unnecessary we chose not to push this tests
into 5.0, but run them manually for this version and later push
their optimized version into 5.1
