DESCRIPTION
===========
Buffer overflow is reported in a lot of code sections
spanning across server, client programs, Regex libraries
etc. If not handled appropriately, they can cause abnormal
behaviour.
ANALYSIS
========
The reported casea are the ones which are likely to result
in SEGFAULT, MEMORY LEAK etc.
FIX
===
- sprintf() has been replaced by my_snprintf() to avoid
buffer overflow.
- my_free() is done after checking if the pointer isn't
NULL already and setting it to NULL thereafter at few
places.
- Buffer is ensured to be large enough to hold the data.
- 'unsigned int' (aka 'uint') is replaced with 'size_t'
to avoid wraparound.
- Memory is freed (if not done so) after its alloced and
used.
- Inserted assert() for size check in InnoDb memcached
code (from 5.6 onwards)
- Other minor changes
IMPLEMENTED IN ALL CLIENT PROGRAMS
Description: Option "enable-cleartext-plugin" is not
available for the following client utilities:-
mysqldump
mysqlimport
mysqlshow
mysqlcheck
Analysis: The unavailability of this option limits the
features like PAM authentication from using the above
mentioned utilities.
Fix: Option "enable-cleartext-plugin" is implemented in the
above mentioned client utilities.
Mysqldump overflows stack buffer when copying table name from commandline arguments resulting in stack corruption and ability to execute arbitrary code.
Fix: Check length of all positional arguments passed to mysqldump is smaller than NAME_LEN.
Note: Mysqldump heavily depends on that database objects (databases, tablespaces, tables, etc) are limited to small size (now it is 64).
Mysqldump overflows stack buffer when copying table name from commandline arguments resulting in stack corruption and ability to execute arbitrary code.
Fix: Check length of all positional arguments passed to mysqldump is smaller than NAME_LEN.
Note: Mysqldump heavily depends on that database objects (databases, tablespaces, tables, etc) are limited to small size (now it is 64).
TO DUMP DATA FROM MYSQL-5.6
Analysis
--------
Dumping mysql-5.6 data using mysql-5.1/mysql-5.5 'myqldump'
utility fails with a syntax error.
Server system variable 'sql_quote_show_create' which quotes the
identifiers is set in the mysqldump utility. The mysldump utility
of mysql-5.1/mysql-5.5 uses deprecated syntax 'SET OPTION' to set
the 'sql_quote_show_create' option. The support for the syntax is
removed in mysql-5.6. Hence syntax error is reported while taking
the dump.
Fix:
---
Changed the 'mysqldump' code to use the syntax
'SET SQL_QUOTE_SHOW_CREATE' to set the 'sql_quote_show_create'
option. That syntax is supported on mysql-5.1, mysql-5.5 and
mysql-5.6.
NOTE: I have not added an mtr test case since it is difficult
to simulate the condition. Also the syntax may not be further
simplified in the future.
MYSQL DB FROM REMOTE 5.0.96 SERVER
Problem: mysqldump tool assumes the existence of
general_log and slow_log tables in the server.
If mysqldump tool executes on a old server where
there are no log tables like these, mysqldump tool
fails.
Analysis: general_log and slow_log tables are added
in the ignore-table list as part of bug-26121 fix
causes bug-45740 (MYSQLDUMP DOESN'T DUMP GENERAL_LOG
AND SLOW_QUERY CAUSES RESTORE PROBLEM). As part of
the bug-45740 fix, mysqldump tool adds create table
queries for these two tables. But the fix assumes
that on all the servers, general_log and slow_log
will be there. If the new mysqldump tool is executed
against a old server where there are no general_log
and slow_log, the mysqldump tool fails with an error
that 'there is no general_log table'.
Fix: When mysqldump tool is trying to retrieve general_log
and slow_log table structures, first the tool should
check their existence of these tables in the server
instead of trying to dump it blindly.
WITH A PORT NUMBER ENCLOSED IN QUOTES
Problem: mysqldump --dump-slave --include-master-host-port
prints the CHANGE MASTER command in the generated logical
backup. The PORT number that is generated with this command
is a string and should be an integer.
Fix: Remove the Enclosed quotes for port number.
Problem: When a view, with a specific character set and collation,
is created on another view with a different character set and collation the
dump restoration results in an illegal mix of collations error.
SOLUTION: To avoid this confusion of collations, the create table datatype
being used is hardcoded as "tinyint NOT NULL". This will not matter as the table
created will be dropped at runtime and specifically tinyint is used to
avoid hitting the row size conflicts.
MYSQLDUMP OUTPUT
Problem: mysqldump when used with option --routines, dumps
all the routines of the specified database into
output. The statements in this output are written
in such a way that they are version safe using C
style version commenting (of the format
/*!<version num> <sql statement>*/). If a semicolon
is present right before closing of the comment in
dump output, it results in a syntax error while
importing.
Solution: Version comments for dumped routines are
specifically to protect the ones older than 5.0.
When the import is done on 5.0 or later versions,
entire create statement gets executed as all the
check conditions at the beginning of the comments
are cleared. Since the trade off is between the
performance of newer versions which are more in
use and protection of very old versions which are
no longer supported, it is proposed that these
comments be removed altogether to maintain
stability of the versions supported.
TABLE DATA IF DUMPS MYSQL DATABA
Problem: If mysqldump is run without --events (or with --skip-events)
it will not dump the mysql.event table's data. This behaviour is inconsistent
with that of --routines option, which does not affect the dumping of
mysql.proc table. According to the Manual, --events (--skip-events) defines,
if the Event Scheduler events for the dumped databases should be included
in the mysqldump output and this has nothing to do with the mysql.event table
itself.
Solution: A warning has been added when mysqldump is used without --events
(or with --skip-events) and a separate patch with the behavioral change
will be prepared for 5.6/trunk.
TABLE DATA IF DUMPS MYSQL DATABA
Problem: If mysqldump is run without --events (or with --skip-events)
it will not dump the mysql.event table's data. This behaviour is inconsistent
with that of --routines option, which does not affect the dumping of
mysql.proc table. According to the Manual, --events (--skip-events) defines,
if the Event Scheduler events for the dumped databases should be included
in the mysqldump output and this has nothing to do with the mysql.event table
itself.
Solution: A warning has been added when mysqldump is used without --events
(or with --skip-events) and a separate patch with the behavioral change
will be prepared for 5.6/trunk.
SHOW 2012 INSTEAD OF 2011
* Added a new macro to hold the current year :
COPYRIGHT_NOTICE_CURRENT_YEAR
* Modified ORACLE_WELCOME_COPYRIGHT_NOTICE macro
to take the initial year as parameter and pick
current year from the above mentioned macro.
executing
The problem is that mysql lacks information about the objects a view
depends on so it can't dump views and tables in the proper order.
Thus it needs to create "stand-in" myisam tables for each view while
dumping the tables that it later drops and replaces with the actual view
view definition.
But since views can have much more columns than an actual table creating
these stand-in tables may be problematic.
There's no way to portably find out how many columns an mysiam table
can have. It's a complicated formula depending on internal server constants.
Thus we can't have a reliable error check without repeating the logic and
the formula inside mysqldump.
1. Changed the type of the columns of the stand-in tables mysqldump
makes to satisfy view dependencies from the original type to smallint
to save on row space.
2. Added a warning on the mysqldump's standard error for a possible
problems replaying the dump file if the columns of a view exceed 1000.
3. Added a test case.
CAUSES RESTORE PROBLEM
Problem Statement:
------------------
mysqldump is not having the dump stmts for general_log and slow_log
tables. That is because of the fix for Bug#26121. Hence, after
dropping the mysql database, and applying the dump by enabling the
logging, "'general_log' table not found" errors are logged into the
server log file.
Analysis:
---------
As part of the fix for Bug#26121, we skipped the dumping of tables
for general_log and slow_log, because the data dump of those tables
are taking LOCKS, which is not allowed for log tables.
Fix:
----
We came up with an approach that instead of taking both meta data
and data dump information for those tables, take only the meta data
dump which doesn't need LOCKS.
As part of fixing the issue we came up with below algorithm.
Design before fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
Design with the fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Explicitly call the 'show create table' for general_log and
slow_log
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
While taking the meta data dump for general_log and slow_log the
"CREATE TABLE" is replaced with "CREATE TABLE IF NOT EXISTS".
This is because we skipped "DROP TABLE" for those tables,
"DROP TABLE" fails for these tables if logging is enabled.
Customer is applying the dump by enabling logging so, if the dump
has "DROP TABLE" it will fail. Hence, removed the "DROP TABLE"
stmts for those tables.
After the fix we could observe "Table 'mysql.general_log'
doesn't exist" errors initially that is because in the customer
scenario they are dropping the mysql database by enabling the
logging, Hence, those errors are expected. Once we apply the
dump which is taken before the "drop database mysql", the errors
will not be there.
CAUSES RESTORE PROBLEM
Problem Statement:
------------------
mysqldump is not having the dump stmts for general_log and slow_log
tables. That is because of the fix for Bug#26121. Hence, after
dropping the mysql database, and applying the dump by enabling the
logging, "'general_log' table not found" errors are logged into the
server log file.
Analysis:
---------
As part of the fix for Bug#26121, we skipped the dumping of tables
for general_log and slow_log, because the data dump of those tables
are taking LOCKS, which is not allowed for log tables.
Fix:
----
We came up with an approach that instead of taking both meta data
and data dump information for those tables, take only the meta data
dump which doesn't need LOCKS.
As part of fixing the issue we came up with below algorithm.
Design before fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
Design with the fix:
1) mysql database is having tables like db, event,... general_log,
... slow_log...
2) Skip general_log and slow_log while preparing the tables list
3) Explicitly call the 'show create table' for general_log and
slow_log
3) Take the TL_READ lock on tables which are present in the table
list and do 'show create table'.
4) Release the lock.
While taking the meta data dump for general_log and slow_log the
"CREATE TABLE" is replaced with "CREATE TABLE IF NOT EXISTS".
This is because we skipped "DROP TABLE" for those tables,
"DROP TABLE" fails for these tables if logging is enabled.
Customer is applying the dump by enabling logging so, if the dump
has "DROP TABLE" it will fail. Hence, removed the "DROP TABLE"
stmts for those tables.
After the fix we could observe "Table 'mysql.general_log'
doesn't exist" errors initially that is because in the customer
scenario they are dropping the mysql database by enabling the
logging, Hence, those errors are expected. Once we apply the
dump which is taken before the "drop database mysql", the errors
will not be there.
routines.
mysqldump in xml mode did not dump routines, events or
triggers.
This patch fixes this issue by fixing the if conditions
that disallowed the dump of above mentioned objects in
xml mode, and added the required code to enable dump
in xml format.
--FLUSH-LOG BREAKS CONSISTENCY
The transaction started by mysqldump gets committed
implicitly when flush-log is specified along with
single-transaction option, and hence can break
consistency.
This is because, COM_REFRESH is executed in order
to flush logs and starting from 5.5 this command
performs an implicit commit.
Fixed by making sure that COM_REFRESH is executed
before the transaction has started and not after it.
Note : This patch triggers following behavioral
changes in mysqldump :
1) After this patch we no longer flush logs before
dumping each database if --single-transaction
option is given like it was done before (in the
absence of --lock-all-tables and --master-data
options).
2) Also, after this patch, we start acquiring
FTWRL before flushing logs in cases when only
--single-transaction and --flush-logs are given.
It becomes safe to use mysqldump with these two
options and without --master-data parameter for
backups.
TOOLS
Backport a fix for Bug 57094 from 5.5.
The following revision was backported:
# revision-id: alexander.nozdrin@oracle.com-20101006150613-ls60rb2tq5dpyb5c
# parent: bar@mysql.com-20101006121559-am1e05ykeicwnx48
# committer: Alexander Nozdrin <alexander.nozdrin@oracle.com>
# branch nick: mysql-5.5-bugteam-bug57094
# timestamp: Wed 2010-10-06 19:06:13 +0400
# message:
# Fix for Bug 57094 (Copyright notice incorrect?).
#
# The fix is to:
# - introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
# to specify copyright notice;
# - replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
# in programs.
("-") IN DATABASE NAMES IN ALTER DATABASE.
mysqldump did not quote database name in 'ALTER DATABASE'
statements in its output. This can further cause a failure
while loading if database name contains a hyphen '-'.
This happened as, while printing the 'ALTER DATABASE'
statements, the database name was not quoted.
Fixed by quoting the database name.
"set optimizer_switch to e or d causes invalid memory writes/valgrind warnings":
due to prefix support, the argument "e" was overwritten with its full value
"engine_condition_pushdown", which caused a buffer overrun.
This was wrong usage of find_type(); other wrong usages are fixed here too.
Please start reading with the comment of typelib.c.
When mysqldump tries to dump information in xml format,
the result does not contain field level comments.
In order to retrieve various informations for a field/column,
mysqldump currently uses 'show fields from <tab>' statement.
The attributes returned by the statement lacks the information
regarding field comments.
Fixed by changing the query to one that probes I_S to retrieve
required field informations, including the field comment.
command line clients.
Command line tools like mysqladmin and mysqldump did not recognize
default-auth and plugin-dir options.
Support for these options was found missing in these command line
tools.
Fixed by adding support for the same.
The fix is to:
- introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
to specify copyright notice;
- replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
in programs.
