Partially backporting MDEV-9874 from 10.2 to 10.0
READ_INFO::read_field() raised the ER_INVALID_CHARACTER_STRING error
when reading an escape character followed by a multi-byte character.
Raising wellformedness errors in READ_INFO::read_field() was wrong,
because the main goal of READ_INFO::read_field() is to *unescape* the
data which was presumably escaped using mysql_real_escape_string(),
using the same character set with the one specified in
"LOAD DATA INFILE ... CHARACTER SET ..." (or assumed by default).
During LOAD DATA, multi-byte characters are not always scanned as a single
entity! In case of escaped data, parts of a multi-byte character can be
scanned on different loop iterations. So the old code erroneously tested
welformedness in the middle of a multi-byte character.
Moreover, the data after unescaping can go into a BLOB field, not a text field.
Wellformedness tests are meaningless in this case.
Ater this patch, wellformedness is only checked later, during
Field::store(str,length,cs) time. The loop that scans bytes only
makes sure to revert the changes made by mysql_real_escape_string().
Note, in some cases users can supply data which did not really go through
mysql_real_escape_string() and was escaped by some other means,
or was not escaped at all. The file reported in this MDEV contains
the string "\ä", which is an example of such improperly escaped data, as
- either there should be two backslashes: "\\ä"
- or there should be no backslashes at all: "ä"
mysql_real_escape_string() could not generate "\ä".
ANALYSIS:
=========
'CREATE TABLE' query with a large value for 'CONNECTION'
string reports an incorrect error.
The length of connection string is stored in .frm in two
bytes (max value= 65535). When the string length exceeds
the max value, the length is truncated to fit the two
bytes limit. Further processing leads to reading only a
part of the string as the length stored is incorrect. The
remaining part of the string is treated as engine type and
hence results in an error.
FIX:
====
We are now restricting the connection string length to 1024.
An appropriate error is reported if the length crosses this
limit.
NOTE:
=====
The 'PASSWORD' table option is documented as unused and
processed within a dead code. Hence it will not cause
similar issue with large strings.
The crash is caused by macro uint3korr() accessing memory (1 byte) past
the end of allocated page. The macro is written such it reads 4 bytes
instead of 3 and discards the value of the last byte.
However, it is not always guaranteed that all uint3korr accesses will be
valid (i.e that the caller allocates an extra byte after the value).
In particular, the tree in Item_func_group_concat does not account for
any extra bytes that it would need for comparison of keys in some cases
(Field_newdate::cmp, Field_medium::cmp)
The fix change uint3korr so it does not access extra bytes.
it was supposed to be used in command-line tools only.
Different fix for 4e5473862e:
Bug#24388746: PRIVILEGE ESCALATION AND RACE CONDITION USING CREATE TABLE
compiled for debugging, when the server goes down
This happens in the following scenario:
- Server gets a shutdown message
- Servers sends error ER_CONNECTION_KILLED to the clients connection
- The client sends a query to the server, before the server has time to
close the connection to the client
- Client reads the ER_CONNECTION_KILLED error message
In the above case, the packet number for the reply is one less than
what the client expected and the client prints "Packets out of order".
Fixed the following way:
- The client accepts now error packages with a packet number
one less than expected.
- To ensure that this issue can be detected early in my_real_read(), error
messages sent to the client are not compressed, even when compressed protocol is used.
During REPAIR TABLE of a MyISAM table, a temporary data file (.TMD)
is created. When repair finishes, this file is renamed to the original
.MYD file. The problem was that during this rename, we copied the
stats from the old file to the new file with chmod/chown. If a user
managed to replace the temporary file before chmod/chown was executed,
it was possible to get an arbitrary file with the privileges of the
mysql user.
This patch fixes the problem by not copying stats from the old
file to the new file. This is not needed as the new file was
created with the correct stats. This fix only changes server
behavior - external utilities such as myisamchk still does
chmod/chown.
No test case provided since the problem involves synchronization
with file system operations.
Addreses are not necessarily between heap_start && heap_end. Malloc
calls using mmap can place pointers outside these bounds. In this case,
we'll warn the user that the query pointer is potentially invalid.
However, we'll attempt to print the data anyway after we're done
printing everything else.
MY_TOSORT_UTF32
This patch is specific for mysql-5.5
ISSUE: When a charater that is larger than possible to
handle is passed to function my_tosort_utf32(), it results
in segmentation fault. In the scenario mentioned in the bug
AES_ENCRYPT function is used which returns large value.
This value is further passed to my_tosort_utf32 function.
This causes to cross array bound for array uni_plane,
resulting in segment violation.
SOLUTION:
This issue has got addressed in 5.6 onward releases
through worklog 2673.
The fix is similar backport of that.
Check for maximum character before accessing the array
uni_plane. In addition to function my_tosort_utf32, the
same potential problem is also present in functions
my_tolower_utf16, my_toupper_utf16, my_tosort_utf16,
my_tolower_utf32, my_toupper_utf32, my_tosort_unicode,
my_tolower_utf8mb4 and my_toupper_utf8mb4.
Fixed these functions as well.
Both aria and myisam storage engines feature a logic path in
thr_find_all_keys that leads to undefined behaviour by bypassing the
initialization code of variables after my_thread_init().
By refactoring the nested logic into a separate function, this problem
is resolved.
Fix test whether process is alive in mysqltest.
Also fix SHUT_RD definition on Windows to be SD_RECEIVE.
SD_BOTH was used instead prior to this patch, and this would
occasionally make mysql_shutdown() fail - when the socket for the current connection
is not able send the COM_SHUTDOWN response anymore.
UNIQUE::~UNIQUE | SQL/UNIQUES.CC:355
Analysis
========
Enabling the sort_buffer_size with a large value
can cause operations utilizing the sort buffer
like DELETE as mentioned in the bug report to
fail. 5.5 and 5.6 versions reports OOM error
while in 5.7+, the server crashes.
While initializing the mem_root for the sort buffer
tree, the block size for the mem_root is determined
from the 'sort_buffer_size' value. This unsigned
long value is typecasted to unsigned int, hence
it becomes zero. Further block_size computation
while initializing the mem_root results in a very
large block_size value. Hence while trying to
allocate a block during the DELETE operation,
an OOM error is reported. In case of 5.7+, the PFS
instrumentation for memory allocation, overshoots
the unsigned value and allocates a block of just
one byte. While trying to free the block of the
mem_root, the original block_size is used. This
triggers the crash since the server tries to free
unallocated memory.
Fix:
====
In order to restrict usage of such unreasonable
sort_buffer_size, the typecast of block size
to 'unsigned int' is removed and hence reports
OOM error across all versions for sizes
exceeding unsigned int range.
"#include <math.h>" has "#define isfinite(X) ..."
while "#include <cmath>" does "#undef isfinite"
in -std=c++11 mode <cmath> is included, we need a workaround
to provide a usable isfinite()
1. check that unused inline functions are removed
2. only allow compilation if they are or if the check if overridden
3. with CMAKE_GENERATOR=Makefiles, use all flags when testing
(e.g. both CMAKE_C_FLAGS and CMAKE_C_FLAGS_DEBUG if
CMAKE_BUILD_TYPE=Debug). This is because
- on Solaris with the SunPro compiler, default CMAKE_C_FLAGS_xxx
values contain -xO2 (for Release and RelWithDebInfo)
and -g (for RelWithDebInfo and Debug)
- proper inlining only works at -xO4 without -g
- so if CMAKE_C_FLAGS has -xO4, inlining would work in
configure.cmake (before this fix) and fail during actual compilation
also remove the outdated check for inline from myu_global.h
cherry-pick f1daf9ce from 10.0 branch
-------------------------------------
Fix build failures caused by new C runtime library
- isnan, snprintf, struct timespec are now defined, attempt to
redefine them leads
- P_tmpdir, tzname are no more defined
- lfind() and lsearch() in lf_hash.c had to be renamed, declaration
conflicts with some C runtime functions with the same name declared in
a header included by stdlib.h
Also fix couple of annoying warnings :
- remove #define NOMINMAX from config.h to avoid "redefined" compiler
warnings(NOMINMAX is already in compile flags)
- disable incremental linker in Debug as well (feature not used much
and compiler crashes often)
Also simplify package building with Wix, require Wix 3.9 or later
(VS2015 is not compatible with old Wix 3.5/3.6)
if mysqld runs as service
The bug is caused by the attempt to write to filedescriptor 2
(STDERR_FILENO), however in case of a service stderr has different fd
(debugging shows fileno(stderr) is 4 after freopen, and -1 before it)
Fixed definition of STDERR_FILENO for Windows to be fileno(stderr).
The check for UCONTEXT in cmake was somehow become broken, disabling the
fallback to ucontext. This caused the non-blocking client API to not be
available for non-x86 platforms, on which no hand-crafted assembler
implementation of my_context is available.
Fix test whether process is alive in mysqltest.
Also fix SHUT_RD definition on Windows to be SD_RECEIVE.
SD_BOTH was used instead prior to this patch, and this would
occasionally make mysql_shutdown() fail - when the socket for the current connection
is not able send the COM_SHUTDOWN response anymore.
Problem Statement
=========
Fix various issues when building MySQL with Visual Studio 2015.
Fix:
=======
- Visual Studio 2015 adds support for timespec. Add check and
related code to use this and only use our replacement if
timespec is not defined.
- Rename lfind/lsearch to my* to avoid redefinition problems.
- Set default value for TMPDIR to "" on Windows as P_tmpdir
no longer exists.
- using VS definition of snprintf if available
- tzname are now renamed to _tzname.
- Added some extra command to rpl_start_stop to ensure that the
IO thread has connected to the master before we shut down the server.
- if signal returns signalhandler_t, use this with the alarm code
- Added missing tests to sys_vars
- Fixed some possible overflow bugs in tabxml.cpp
Fix build failures caused by new C runtime library
- isnan, snprintf, struct timespec are now defined, attempt to
redefine them leads
- P_tmpdir, tzname are no more defined
- lfind() and lsearch() in lf_hash.c had to be renamed, declaration
conflicts with some C runtime functions with the same name declared in
a header included by stdlib.h
Also fix couple of annoying warnings :
- remove #define NOMINMAX from config.h to avoid "redefined" compiler
warnings(NOMINMAX is already in compile flags)
- disable incremental linker in Debug as well (feature not used much
and compiler crashes often)
Also simplify package building with Wix, require Wix 3.9 or later
(VS2015 is not compatible with old Wix 3.5/3.6)
Fixed unclean prototype declaration. According to ISO/IEC 9899:TC2:
...
10. The special case of an unnamed parameter of type void as the only item in
the list specifies that the function has no parameters.
...
14. An identifier list declares only the identifiers of the parameters of the
function. An empty list in a function declarator that is part of a
definition of that function specifies that the function has no parameters.
The empty list in a function declarator that is not part of a definition of
that function specifies that no information about the number or types of the
parameters is supplied. 124)
...
6.11.6 Function declarators
The use of function declarators with empty parentheses (not prototype-format
parameter type declarators) is an obsolescent feature.
...
Patch contributed by Michal Hrusecky.
of more than 45G with a key_cache_block_size of 1024 or less.
The problem was that some of the arguments to my_multi_malloc() got to be
more than 4G.
Fix:
- Inntroduced my_multi_malloc_large() that can handle big regions.
- Changed MyISAM and Aria key caches to use my_multi_malloc_large().
I didn't change the default my_multi_malloc() as this would be a too big
patch and we don't allocate 4G blocks anywhere else.
Description: The newest RHEL/CentOS/SL 6.6 openssl package
(1.0.1e-30.el6_6.9; published around 6/4/2015) contains a fix for
LogJam. RedHat's fix for this was to limit the use
of any SSL DH key sizes to a minimum of 768 bits. This breaks any
DHE SSL ciphers for MySQL clients as soon as you install the
openssl update, since in vio/viosslfactories.c, the default
DHPARAM is a 512 bit one. This cannot be changed in
configuration/runtime; and needs a recompile. Because of this the
client connection with --ssl-cipher=DHE-RSA-AES256-SHA is not
able to connect the server.
Analysis: Openssl has changed Diffie-Hellman key from the 512 to
1024 due to some reasons(please see the details at
http://openssl.org/news/secadv_20150611.txt) Because of this the client
with DHE cipher is failing to connect the server. This change took
place from the openssl-1.0.1n onwards.
Fix: Similar bug fix is already pushed to mysql-5.7 under bug#18367167.
Hence we backported the same fix to mysql-5.5 and mysql-5.6.
in cmake tests let's treat clang like gcc (same options,
same builtins) in many cases.
* don't check the compiler when
* testing for -fvisibility=hidden support
* testing for HAVE_ABI_CXA_DEMANGLE
* testing for HAVE_GCC_ATOMIC_BUILTINS
* when removing options with string(replace)
* when running ${CC} --version (ignore the error instead)
* run ABI checks for clang
* use "canonical" gcc flags for clang
* fix groonga too
Also:
* add cmake detection for gcc __atomic_* builtins. they might be
supported (__ATOMIC_SEQ_CST is defined), but not for all operand
sizes. In particular, 64-bit atomic load is problematic on i386
* cache check results for Windows
* remove the test for HAVE_CXXABI_H (HAVE_ABI_CXA_DEMANGLE is
suffifient)
