This commit is based on the work of Michal Schorm, rebased on the
earliest MariaDB version.
Th command line used to generate this diff was:
find ./ -type f \
-exec sed -i -e 's/Foundation, Inc., 59 Temple Place, Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/Foundation, Inc. 59 Temple Place.* Suite 330, Boston, /Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, /g' {} \; \
-exec sed -i -e 's/MA.*.....-1307.*USA/MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/Foundation, Inc., 59 Temple/Foundation, Inc., 51 Franklin/g' {} \; \
-exec sed -i -e 's/Place, Suite 330, Boston, MA.*02111-1307.*USA/Street, Fifth Floor, Boston, MA 02110-1335 USA/g' {} \; \
-exec sed -i -e 's/MA.*.....-1307/MA 02110-1335/g' {} \;
Problem:
========
The mysqlbinlog tool is leaking memory, causing failures in various tests when
compiling and testing with AddressSanitizer or LeakSanitizer like this:
cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_ASAN:BOOL=ON /path/to/source
make -j$(nproc)
cd mysql-test
ASAN_OPTIONS=abort_on_error=1 ./mtr --parallel=auto rpl.rpl_row_mysqlbinlog
CURRENT_TEST: rpl.rpl_row_mysqlbinlog
Direct leak of 112 byte(s) in 1 object(s) allocated from:
#0 0x4eff87 in __interceptor_malloc (/dev/shm/5.5/client/mysqlbinlog+0x4eff87)
#1 0x60eaab in my_malloc /mariadb/5.5/mysys/my_malloc.c:41:10
#2 0x5300dd in Log_event::read_log_event(char const*, unsigned int, char const**,
Format_description_log_event const*, char) /mariadb/5.5/sql/log_event.cc:1568:
#3 0x564a9c in dump_remote_log_entries(st_print_event_info*, char const*)
/mariadb/5.5/client/mysqlbinlog.cc:1978:17
Analysis:
========
'mysqlbinlog' tool is being used to read binary log events from a remote server.
While reading binary log, if a fake rotate event is found following actions are
taken.
If 'to-last-log' option is specified, then fake rotate event is processed.
In the absence of 'to-last-log' skip the fake rotate event.
In this skipped case the fake rotate event object is not getting cleaned up
resulting in memory leak.
Fix:
===
Cleanup the fake rotate event.
This issues is already fixed in MariaDB 10.0.23 and higher versions as part of
commit c3018b0ff4
Disable LOAD DATA LOCAL INFILE suport by default and
auto-enable it for the duration of one query, if the query
string starts with the word "load". In all other cases the application
should enable LOAD DATA LOCAL INFILE support explicitly.
Close connection handler on connection failure. This fixes 14 failing tests in
main suite under clang+ASAN build.
ASAN report for main.connect looks like this:
=================================================================
==25495==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 146280 byte(s) in 115 object(s) allocated from:
#0 0x4fba47 in calloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:138
#1 0x5a7a02 in mysql_init /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:977:26
#2 0x570a7a in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6096:26
#3 0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9
#4 0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
Indirect leak of 7065600 byte(s) in 115 object(s) allocated from:
#0 0x4fb80f in __interceptor_malloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:129
#1 0x637a83 in my_context_init /work/mariadb/libmariadb/libmariadb/ma_context.c:367:23
#2 0x59fd16 in mysql_optionsv /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:2738:9
#3 0x5bc1d4 in mysql_options /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:3242:10
#4 0x570b94 in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6103:7
#5 0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9
#6 0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
Indirect leak of 940240 byte(s) in 115 object(s) allocated from:
#0 0x4fb80f in __interceptor_malloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:129
#1 0x64386e in ma_init_dynamic_array /work/mariadb/libmariadb/libmariadb/ma_array.c:49:31
#2 0x649ead in _hash_init /work/mariadb/libmariadb/libmariadb/ma_hash.c:52:7
#3 0x5a3080 in mysql_optionsv /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:2938:13
#4 0x5bc20c in mysql_options4 /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:3248:10
#5 0x56f63b in connect_n_handle_errors(st_command*, st_mysql*, char const*, char const*, char const*, char const*, int, char const*) /work/mariadb/client/mysqltest.cc:5874:3
#6 0x57146b in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6193:7
#7 0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9
#8 0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
...
Closes#809
client/mysql.cc: In function void build_completion_hash(bool, bool):
client/mysql.cc:2674:37: error: invalid conversion from char to char* [-fpermissive]
field_names[i][num_fields*2]= '\0';
--BINARY-AS-HEX, --NO-BEEP
DESCRIPTION:
============
mysql uses -b as the short-option form for two different
long options i.e. no-beep and binary-as-hex. This can
result in unintended results if the short form -b is used
instead of the specific long option name.
FIX:
====
-b will now be used for one long option only i.e --no-beep.
The option binary-as-hex will not have any short option and
should be provided as a complete name.
client/mysql.cc: In function void build_completion_hash(bool, bool):
client/mysql.cc:2674:37: error: invalid conversion from char to char* [-fpermissive]
field_names[i][num_fields*2]= '\0';
^~~~
The order of outputting stored procedures is important. Stored
procedures must be available on view creation, for views which make use
of them. Make sure to print them before outputting tables.
The bug arises when one uses --auto-generate-sql-guid-primary (and
--auto-generate-sql-secondary-indexes) with mysqlslap and also have
sql_mode=STRICT_TRANS_TABLE.
When using this option, mysqlslap should create a column with varchar(36),
but it appears to create it as a varchar(32) only. Then if one has
sql_mode=STRICT_TRANS_TABLES, it throws an error, like:
mysqlslap: Cannot run query INSERT INTO t1 VALUES (...)
ERROR : Data too long for column 'id' at row 1
Upstream bug report: BUG#80329.
Description: Mysqldump utility fails for specific clauses
used with the option, 'where'.
Analysis:- Method, "fix_identifier_with_newline()" that
prefixes all occurrences of newline char ('\n') in incoming
buffer does not verify the size of the buffer. The buffer in
which the incoming buffer is copied is limited to 2048 bytes
and the method does not try to allocate additional memory
for larger incoming buffers.
Fix:- Method, "fix_identifier_with_newline()" is modified
to fix this issue.
DESCRIPTION:
===========
The bug is related to incorrect parsing of SQL queries
when typed in on the CLI. The incorrect parsing can
result in unexpected results.
ANALYSIS:
========
The scenarios mainly happens for identifier names
with a typical combination of backslashes and backticks.
The incorrect parsing can either result in executing
additional queries or can result in query truncation.
This can impact mysqldump as well.
FIX:
===
The fix makes sure that such identifier names are
correctly parsed and a proper query is sent to the
server for execution.
(cherry picked from commit 31a372aa1c2b93dc75267d1f05a7f7fca6080dc0)
DESCRIPTION:
===========
The bug is related to incorrect parsing of SQL queries
when typed in on the CLI. The incorrect parsing can
result in unexpected results.
ANALYSIS:
========
The scenarios mainly happens for identifier names
with a typical combination of backslashes and backticks.
The incorrect parsing can either result in executing
additional queries or can result in query truncation.
This can impact mysqldump as well.
FIX:
===
The fix makes sure that such identifier names are
correctly parsed and a proper query is sent to the
server for execution.
field_names[x][y] is a pointer
client/mysql.cc: In function 'void build_completion_hash(bool, bool)':
client/mysql.cc:2855:37: error: invalid conversion from 'char' to 'char*' [-fpermissive]
field_names[i][num_fields*2]= '\0';
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>
When there are quotes in the USE statement, the mysql client does
not correctly escape them.
The USE statement is processed line by line from the client's parser,
and cannot handle multi-line commands as the server.
The fix is to escape the USE parameters whenever quotes are used.
CLIENT (CONTRIBUTION)
DESCRIPTION:
============
Binary data should be printed as hex in the mysql client
when the option binary-as-hex is enabled.
ANALYSIS:
=========
The fix deals only with mysql command line client.
It does not change, at all, the data sent to the
applications. Printing binary data as hex also
allows to use the output in the where clause
of the query.
FIX:
====
A new option 'binary-as-hex' is introduced to print the
binary contents as hex in the mysql client. The option
is disabled by default. When the option is enabled, we
convert the binary data to hex before printing the
contents irrespective of whether it is in tabular,
xml or html format.
While writing comments if database object names has a new
line character, then next line is considered a command, rather
than a comment.
This patch fixes the way comments are constructed in mysqldump.
(cherry picked from commit 1099f9d17b1c697c2760f86556f5bae7d202b444)
While writing comments if database object names has a new
line character, then next line is considered a command, rather
than a comment.
This patch fixes the way comments are constructed in mysqldump.
MYSQL_OPT_SSL_MODE option introduced.
It is set in case of --ssl-mode=REQUIRED and permits only SSL connection.
(cherry picked from commit 3b2d28578c526f347f5cfe763681eff365731f99)
Backport the fix to 5.5, because it fails there too
The patch fixes two test failures:
- on slow builders, sometimes a connection attempt which should
fail due to the exceeded number of thread_pool_max_threads
actually succeeds;
- on even slow builders, MTR sometimes cannot establish the
initial connection, and check-testcase fails prior to the
test start
The problem with check-testcase was caused by connect-timeout=2
which was set for all clients in the test config file. On slow
builders it might be not enough.
There is no way to override it for the pre-test check, so it needed
to be substantially increased or removed.
The other problem was caused by a race condition between sleeps
that the test performs in existing connections and the connect
timeout for the connection attempt which was expected to fail.
If sleeps finished before the connect-timeout was exceeded, it
would allow the connection to succeed.
To solve each problem without making the other one worse,
connect-timeout should be configured dynamically during the test.
Due to the nature of the test (all connections must be busy
at the moment when we need to change the timeout, and cannot execute
SET GLOBAL ...), it needs to be done independently from the server.
The solution:
- recognize 'connect_timeout' as a connection option in mysqltest's
"connect" command;
- remove connect-timeout from the test configuration file;
- use the new connect_timeout option for those connections which
are expected to fail;
- re-arrange the test flow to allow running a huge SLEEP
without affecting the test execution time (because it would be
interrupted after the main test flow is finished).
The test is still subject to false negatives, e.g. if the connection
fails due to timeout rather than due to the exceeded number of
allowed threads, or if the connection on extra port succeeds due
to a race condition and not because the special logic for the extra
port. But those false negatives have always been possible there
on slow builders, they should not be critical because faster builders
should catch such failures if they appear.
Conflicts:
client/mysqltest.cc
mysql-test/r/pool_of_threads.result
mysql-test/t/pool_of_threads.test
mysqldump/mysql_install_db.exe fail
The bug is described in
https://connect.microsoft.com/VisualStudio/Feedback/Details/1902345
When reading from a pipe in text mode, using CRT function such as fread(),
some newlines may be lost. Workaround is to use binary mode on reading side
and if necessary, replace \r\n with \n.
RUNNING FIREWALL
mysqladmin shutdown will try to extract the server's pid file before executing
the actual shutdown command.
It will do that by executing a SHOW VARIABLES query and processing the result.
However if that query fails it print a (somewhat confusing) error mesasage
and will still continue to do the shutdown command.
If that passes then the mysqladmin user will get an error but the shutdown will
still be successful.
This is confusing so the error message text is changed to say that this is a
non-fatal error and execution continues.
No test case added since it'd require a selective query failure device that's
not available in 5.5.