LOAD DATA AT MASTER.
Revert "BUG#23080148 - BACKPORT BUG 14653594 AND BUG 20683959 TO"
This reverts commit 1d31f5b3090d129382b50b95512f2f79305715a1.
The commit causes replication incompatibility between minor revisions
and based on discussion with Srinivasarao, the patch is reverted.
MYSQL-5.5
The bug asks for a backport of bug#1463594 and bug#20682959. This
is required because of the fact that if replication is enabled, master
transaction can commit whereas slave can't commit due to not exact
'enviroment'. This manifestation is seen in bug#22024200.
LOCAL AND IMPORT ERRORS
Description:
-----------
This bug happens due to the fact that current algorithm is designed
that in the case of LOCAL load of data, in case of the error, the
remaining part of the file is read in order to return the proper
error message to the client side.
But, the problem with current implementation is that data stream
for the client side is cleared only in the case where line delimiters
exist, which is not a case with, for example fixed width
fields.
Fix:
----
Ported patch provided by Sinisa Milivojevic n bug report for this
issue to 5.5+ versions.
As part of this patch code is changed to clear the data stream
by calling new member function "READ_INFO::skip_data_till_eof".
-----------
After compiling from source, during make test I got the following error:
test main.loaddata failed with error
CURRENT_TEST: main.loaddata
mysqltest: At line 592: query 'LOAD DATA INFILE 'tmpp.txt' INTO TABLE t1
CHARACTER SET ucs2
(@b) SET a=REVERSE(@b)' failed: 1115: Unknown character set: 'ucs2'
I noticed other tests are skipped because of no ucs2
main.mix2_myisam_ucs2 [ skipped ] Test requires:'
have_ucs2'
Should main.loaddata be skipped if there is no ucs2
How To Repeat:
-------------
Run make test on compiled source that doesn't have ucs2
Suggested fix:
-------------
the failing piece of the test should be moved from mysql-test/t/loaddata.test to
mysql-test/t/ctype_ucs.test.
-----------
After compiling from source, during make test I got the following error:
test main.loaddata failed with error
CURRENT_TEST: main.loaddata
mysqltest: At line 592: query 'LOAD DATA INFILE 'tmpp.txt' INTO TABLE t1
CHARACTER SET ucs2
(@b) SET a=REVERSE(@b)' failed: 1115: Unknown character set: 'ucs2'
I noticed other tests are skipped because of no ucs2
main.mix2_myisam_ucs2 [ skipped ] Test requires:'
have_ucs2'
Should main.loaddata be skipped if there is no ucs2
How To Repeat:
-------------
Run make test on compiled source that doesn't have ucs2
Suggested fix:
-------------
the failing piece of the test should be moved from mysql-test/t/loaddata.test to
mysql-test/t/ctype_ucs.test.
-----------
After compiling from source, during make test I got the following error:
test main.loaddata failed with error
CURRENT_TEST: main.loaddata
mysqltest: At line 592: query 'LOAD DATA INFILE 'tmpp.txt' INTO TABLE t1
CHARACTER SET ucs2
(@b) SET a=REVERSE(@b)' failed: 1115: Unknown character set: 'ucs2'
I noticed other tests are skipped because of no ucs2
main.mix2_myisam_ucs2 [ skipped ] Test requires:'
have_ucs2'
Should main.loaddata be skipped if there is no ucs2
How To Repeat:
-------------
Run make test on compiled source that doesn't have ucs2
Suggested fix:
-------------
the failing piece of the test should be moved from mysql-test/t/loaddata.test to
mysql-test/t/ctype_ucs.test.
Some multibyte sequences could be considered by my_mbcharlen() functions
as multibyte character but more exact my_ismbchar() does not think so.
In such a case this multibyte sequences is pushed into 'stack' buffer which
is too small to accommodate the sequence.
The fix is to allocate stack buffer in
compliance with max character length.
and reverse() function
3 problems fixed :
1. The reported problem : caused by incorrect parsing of
the file as ucs data resulting in wrong length of the parsed
string. Fixed by truncating the invalid trailing bytes
(non-complete multibyte characters) when reading from the file
2. LOAD DATA when reading from a proper UCS2 file wasn't
recognizing the new line characters. Fixed by first looking
if a byte is a new line (or any other special) character before
reading it as a part of a multibyte character.
3. When using user variables to hold the column data in LOAD
DATA the character set of the user variable was set incorrectly
to the database charset. Fixed by setting it to the charset
specified by LOAD DATA (if any).
Conflicts:
Text conflict in mysql-test/r/explain.result
Text conflict in mysql-test/t/explain.test
Text conflict in sql/net_serv.cc
Text conflict in sql/sp_head.cc
Text conflict in sql/sql_priv.h
Iterative patch improvement. Previously committed patch
caused wrong result on Windows. The previous patch also
broke secure_file_priv for symlinks since not all file
paths which must be compared against this variable are
normalized using the same norm.
The server variable opt_secure_file_priv wasn't
normalized properly and caused the operations
LOAD DATA INFILE .. INTO TABLE ..
and
SELECT load_file(..)
to do different interpretations of the
--secure-file-priv option.
The patch moves code to the server initialization
routines so that the path always is normalized
once and only once.
It was also intended that setting the option
to an empty string should be equal to
lifting all previously set restrictions. This
is also fixed by this patch.
Conflicts:
Text conflict in configure.in
Text conflict in dbug/dbug.c
Text conflict in mysql-test/r/ps.result
Text conflict in mysql-test/t/ps.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/ha_ndbcluster.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/sql_plugin.cc
Text conflict in sql/sql_table.cc
on LOAD DATA
Two problems :
1. LOAD DATA was not checking for SQL errors and was sending an OK
packet even when there were errors reported already. Fixed to check for
SQL errors in addition to the error conditions already detected.
2. There was an over-ambitious assert() on the server to check if the
protocol is always followed by the client. This can cause crashes on
debug servers by clients not completing the protocol exchange for some
reason (e.g. --send command in mysqltest). Fixed by keeping the assert
only on client side, since the server always completes the protocol
exchange.
Conflicts:
Text conflict in mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
Text conflict in sql/log.cc
Text conflict in sql/set_var.cc
Text conflict in sql/sql_class.cc
function on windows
When making sure that the directory path ends up with a
slash/backslash we need to check for the correct length of
the buffer and trim at the appropriate location so we don't
write past the end of the buffer.
col equal to itself!
There's no need to copy the value of a field into itself.
While generally harmless (except for some performance penalties)
it may be dangerous when the copy code doesn't expect this.
Fixed by checking if the source field is the same as the destination
field before copying the data.
Note that we must preserve the order of assignment of the null
flags (hence the null_value assignment addition).
function on windows
When making sure that the directory path ends up with a
slash/backslash we need to check for the correct length of
the buffer and trim at the appropriate location so we don't
write past the end of the buffer.
to string conversions and vice versa"
Initial import of the dtoa.c code and custom wrappers around it
to allow its usage from the server code.
Conversion of FLOAT/DOUBLE values to DECIMAL ones or strings
and vice versa has been significantly reworked. As the new
algoritms are more precise than the older ones, results of such
conversions may not always match those obtained from older
server versions. This in turn may break compatibility for some
applications.
This patch also fixes the following bugs:
- bug #12860 "Difference in zero padding of exponent between
Unix and Windows"
- bug #21497 "DOUBLE truncated to unusable value"
- bug #26788 "mysqld (debug) aborts when inserting specific
numbers into char fields"
- bug #24541 "Data truncated..." on decimal type columns
without any good reason"
Problem 1: main.loaddata tried to trigger an error caused by
reading files outside the vardir, by reading itself. However,
if loaddata.test is not world-readable (e.g., umask=0077),
then another error is triggered.
Fix 1: allow the other error too.
Problem 2: rpl_slave_skip and rpl_innodb_mixed_dml tried to
copy a file from mysql-test/suite/rpl/data to mysql-test/var
and then read it. That failed too if umask=0077, since the
file would not become world-readable.
Fix 2: move the files from mysql-test/suite/rpl/data to
mysql-test/std_data and update tests accordingly. Remove
the directory mysql-test/suite/rpl/data.
The problem was that LOAD DATA code (sql_load.cc) didn't take into
account that there may be items, representing references to other
columns. This is a usual case in views. The crash happened because
Item_direct_view_ref was casted to Item_user_var_as_out_param,
which is not a base class.
The fix is to
1) Handle references properly;
2) Ensure that an item is treated as a user variable only when
it is a user variable indeed;
3) Report an error if LOAD DATA is used to load data into
non-updatable column.
The SELECT INTO OUTFILE FIELDS ENCLOSED BY digit or minus sign,
followed by the same LOAD DATA INFILE statement, used wrond encoding
of non-string fields contained the enclosed character in their text
representation.
Example:
SELECT 15, 9 INTO OUTFILE 'text' FIELDS ENCLOSED BY '5';
Old encoded result in the text file:
5155 595
^ was decoded as the 1st enclosing character of the 2nd field;
^ was skipped as garbage;
^ ^ was decoded as a pair of englosing characters of the 1st field;
^ was decoded as traling space of the first field;
^^ was decoded as a doubled enclosed character.
New encoded result in the text file:
51\55 595
^ ^ pair of enclosing characters of the 1st field;
^^ escaped enclosed character.
The `SELECT 'r' INTO OUTFILE ... FIELDS ENCLOSED BY 'r' ' statement
encoded the 'r' string to a 4 byte string of value x'725c7272'
(sequence of 4 characters: r\rr).
The LOAD DATA statement decoded this string to a 1 byte string of
value x'0d' (ASCII Carriage Return character) instead of the original
'r' character.
The same error also happened with the FIELDS ENCLOSED BY clause
followed by special characters: 'n', 't', 'r', 'b', '0', 'Z' and 'N'.
NOTE 1: This is a result of the undocumented feature: the LOAD DATA INFILE
recognises 2-byte input sequences like \n, \t, \r and \Z in addition
to documented 2-byte sequences: \0 and \N. This feature should be
documented (here backspace character is a default ESCAPED BY character,
in the real-life example it may be any ESCAPED BY character).
NOTE 2, changed behaviour:
Now the `SELECT INTO OUTFILE' statement with the `FIELDS ENCLOSED BY'
clause followed by one of: 'n', 't', 'r', 'b', '0', 'Z' or 'N' characters
encodes this special character itself by doubling it ('r' --> 'rr'),
not by prepending it with an escape character.
TIMESTAMP field when no value has been provided.
The LOAD DATA sets the current time in the TIMESTAMP field with
CURRENT_TIMESTAMP default value when the field is detected as a null.
But when the LOAD DATA command loads data from a file that doesn't contain
enough data for all fields then the rest of fields are simply set to null
without any check. This leads to no value being inserted to such TIMESTAMP
field.
Now the read_sep_field() and the read_fixed_length() functions set current
time to the TIMESTAMP field with CURRENT_TIMESTAMP default value in all cases
when a NULL value is loaded to the field.
