'SYSTEM LOCK' IN PROCESSLIST
Analysis
=========
Show processlist shows 'System Lock' in 'State' field while
LOAD DATA INFILE is running.
thd->proc_info update is missing in LOAD DATA INFILE path.
Thus any request will get last unpdated status from lock_table()
during open_table().
Fix:
=======
Update state information from LOAD DATA INFILE path.
DESCRIPTION
===========
Inability of mysql LOAD XML command to handle empty XML
tags i.e. <row><tag/></row>. Also the behaviour is wrong
and (different than above) when there is a space in empty
tag i.e. <row><tag /></row>
ANALYSIS
========
In read_xml() the case where we encounter a close tag ('/')
we're decreasing the 'level' blindly which is wrong.
Actually when its an without-space-empty-tag (succeeding
char is '>'), we need to skip the decrement. In other words
whenever we hit a close tag ('/'), decrease the 'level'
only when (i) It's not an (without space) empty tag i.e.
<tag/> or, (ii) It is of format <row col="val" .../>
FIX
===
The switch case for '/' is modified. We've removed the
blind decrement of 'level'. We do it only when its not an
without-space-empty-tag. Also we are setting 'in_tag' to
false to let program know that we're done reading current
tag (required in the case of format <row col="val" .../>)
Bug#18396916 MAIN.OUTFILE_LOADDATA TEST FAILS ON ARM, AARCH64, PPC/PPC64
The recorded results for the failing tests were wrong.
They were introduced by the patch for
Bug#30946 mysqldump silently ignores --default-character-set when used with --tab
Correct results were returned for platforms where 'char' is implemented as unsigned.
This was reported as
Bug#46895 Test "outfile_loaddata" fails (reproducible)
Bug#11755168 46895: TEST "OUTFILE_LOADDATA" FAILS (REPRODUCIBLE)
The patch for that bug fixed only parts of the problem,
leaving the incorrect results in the .result file.
Solution: use 'uchar' for field_terminator and line_terminator on all platforms.
Also: remove some un-necessary casts, leaving the ones we actually need.
LOCAL AND IMPORT ERRORS
Description:
-----------
This bug happens due to the fact that current algorithm is designed
that in the case of LOCAL load of data, in case of the error, the
remaining part of the file is read in order to return the proper
error message to the client side.
But, the problem with current implementation is that data stream
for the client side is cleared only in the case where line delimiters
exist, which is not a case with, for example fixed width
fields.
Fix:
----
Ported patch provided by Sinisa Milivojevic n bug report for this
issue to 5.5+ versions.
As part of this patch code is changed to clear the data stream
by calling new member function "READ_INFO::skip_data_till_eof".
REPEATED TWICE IN BINLOG
Problem:
=======
If LOAD DATA ... SET ... is used the last argument of SET is
repeated twice in replication binlog.
Analysis:
========
LOAD DATA statements are reconstructed once again before
they are written to the binary log. When SET clauses are
specified as part of LOAD DATA statement, these SET clause
user command strings need to be stored in order to rebuild
the original user command. During parsing each column and
the value in the SET command are stored in two differenet
lists. All the values are stored in a string list.
When SET expression has more than one value as shown in the
following example:
SET a = @a, b = CONCAT(@b, '| 123456789');
Parser extracts values in the following manner i.e Item name
, value string, actual length of the value of the item with
in the string.
Item a:
Value for a:"= @a, b = CONCAT(@b, '| 123456789')
str_length = 4
Item b:
Value for b:"= CONCAT(@b, '| 123456789')
str_length = 27
During reconstructing the LOAD DATA command the above
strings are retrived as it is and appended to the LOAD DATA
statement. Hence it becomes as shown below.
SET `a`= @a, b = CONCAT(@b, '| 123456789'),
`b`= CONCAT(@b, '| 123456789')
Fix:
===
During reconstruction of SET command, retrieve exact item
value string rather than reading the entire string.
LOAD DATA CAN CAUSE SQL INJECTION
Problem:
=======
A long SET expression in LOAD DATA is incorrectly truncated
when written to the binary log.
Analysis:
========
LOAD DATA statements are reconstructed once again before
they are written to the binary log. When SET clauses are
specified as part of LOAD DATA statement, these SET clause
user command strings need to be stored as it is inorder to
reconstruct the original user command. At present these
strings are stored as part of SET clause item tree's
top most Item node's name itself which is incorrect. As an
Item::name can be of MAX_ALIAS_NAME (256) size. Hence the
name will get truncated to "255".
Because of this the rewritten LOAD DATA statement will be
terminated incorrectly. When this statment is read back by
the mysqlbinlog tool it reads a starting single quote and
continuos to read till it finds an ending quote. Hence any
statement written post ending quote will be considered as
a new statement.
Fix:
===
As name field has length restriction the string value
should not be stored in Item::name. A new String list is
maintained to store the SET expression values and this list
is read during reconstrution.
QUOTING IN REPLICATION
Problem: Misquoting or unquoted identifiers may lead to
incorrect statements to be logged to the binary log.
Fix: we use specialized functions to append quoted identifiers in
the statements generated by the server.
GCC 4.6 has new -Wunused-but-set-variable flag, which is enabled
by -Wall, that causes GCC to emit a warning whenever a local variable
is assigned to, but otherwise unused (aside from its declaration).
Since the maintainer mode uses -Wall and -Werror, source code which
triggers these warnings will be rejected. That is, these warnings
become hard errors.
The solution is to fix the code which triggers these specific warnings.
In most of the cases, this is a welcome cleanup as code which triggers
this warning is probably dead anyway.
If LOAD DATA INFILE featured a SET clause, the name=value pairs
would be regenerated using item::print. Unfortunately, that code
is mostly optimized for EXPLAIN EXTENDED output and such, and can
not be relied on to return valid SQL.
We now name each value its original, user-supplied form and use
that to create LOAD DATA INFILE statements for statement-based
replication.
USING '..' ON WINDOWS
Backport of the fix to 5.0 (to be null-merged to 5.1).
Moved the test into the main test suite.
Made mysql-test-run.pl to not use symlinks for sdtdata as the symlinks
are now properly recognized by secure_file_priv.
Made sure the paths in load_file(), LOAD DATA and SELECT .. INTO OUTFILE
that are checked against secure_file_priv in a correct way similarly to 5.1
by the extended is_secure_file_path() backport before the comparison.
Added an extensive test with all the variants of upper/lower case,
slash/backslash and case sensitivity.
Added few comments to the code.
Some multibyte sequences could be considered by my_mbcharlen() functions
as multibyte character but more exact my_ismbchar() does not think so.
In such a case this multibyte sequences is pushed into 'stack' buffer which
is too small to accommodate the sequence.
The fix is to allocate stack buffer in
compliance with max character length.
On windows, an #endif in a wrong place was causing an early
return from mysql_load and thus the LOAD DATA LOCAL was not
executed. This problem was fixed by moving the #endif to the
right place.
The following code was missing
if ((stat_info.st_mode & S_IFIFO) == S_IFIFO)
is_fifo = 1;
which is required to properly configure and read from the
IO_CACHE when a named pipe is used. So it was re-introduced
before the #endif.
"load data infile .." allowed for access to
unautohorized tables.
Due to a faulty if-statement it was possible to
circumvent the secure_file_priv restriction.
Manual merge from mysql-5.1-bugteam into mysql-5.5-bugteam.
Conflicts
=========
Text conflict in sql/log.cc
Text conflict in sql/log.h
Text conflict in sql/slave.cc
Text conflict in sql/sql_parse.cc
Text conflict in sql/sql_priv.h
when generating new name.
If find_uniq_filename returns an error, then this error is not
being propagated upwards, and execution does not report error to
the user (although a entry in the error log is generated).
Additionally, some more errors were ignored in new_file_impl:
- when writing the rotate event
- when reopening the index and binary log file
This patch addresses this by propagating the error up in the
execution stack. Furthermore, when rotation of the binary log
fails, an incident event is written, because there may be a
chance that some changes for a given statement, were not properly
logged. For example, in SBR, LOAD DATA INFILE statement requires
more than one event to be logged, should rotation fail while
logging part of the LOAD DATA events, then the logged data would
become inconsistent with the data in the storage engine.
With statement- or mixed-mode logging, "LOAD DATA INFILE" queries
are written to the binlog using special types of log events.
When mysqlbinlog reads such events, it re-creates the file in a
temporary directory with a generated filename and outputs a
"LOAD DATA INFILE" query where the filename is replaced by the
generated file. The temporary file is not deleted by mysqlbinlog
after termination.
To fix the problem, in mixed mode we go to row-based. In SBR, we
document it to remind user the tmpfile is left in a temporary
directory.
With statement- or mixed-mode logging, "LOAD DATA INFILE" queries
are written to the binlog using special types of log events.
When mysqlbinlog reads such events, it re-creates the file in a
temporary directory with a generated filename and outputs a
"LOAD DATA INFILE" query where the filename is replaced by the
generated file. The temporary file is not deleted by mysqlbinlog
after termination.
To fix the problem, in mixed mode we go to row-based. In SBR, we
document it to remind user the tmpfile is left in a temporary
directory.
With statement- or mixed-mode logging, "LOAD DATA INFILE" queries
are written to the binlog using special types of log events.
When mysqlbinlog reads such events, it re-creates the file in a
temporary directory with a generated filename and outputs a
"LOAD DATA INFILE" query where the filename is replaced by the
generated file. The temporary file is not deleted by mysqlbinlog
after termination.
To fix the problem, in mixed mode we go to row-based. In SBR, we
document it to remind user the tmpfile is left in a temporary
directory.
With statement- or mixed-mode logging, "LOAD DATA INFILE" queries
are written to the binlog using special types of log events.
When mysqlbinlog reads such events, it re-creates the file in a
temporary directory with a generated filename and outputs a
"LOAD DATA INFILE" query where the filename is replaced by the
generated file. The temporary file is not deleted by mysqlbinlog
after termination.
To fix the problem, in mixed mode we go to row-based. In SBR, we
document it to remind user the tmpfile is left in a temporary
directory.
Fix warnings flagged by the new warning option -Wunused-but-set-variable
that was added to GCC 4.6 and that is enabled by -Wunused and -Wall. The
option causes a warning whenever a local variable is assigned to but is
later unused. It also warns about meaningless pointer dereferences.
