routines.
mysqldump in xml mode did not dump routines, events or
triggers.
This patch fixes this issue by fixing the if conditions
that disallowed the dump of above mentioned objects in
xml mode, and added the required code to enable dump
in xml format.
client/mysqldump.c:
BUG#11760384 - 52792: mysqldump in XML mode does not dump
routines.
Fixed some if conditions to allow execution of dump methods
for xml and further added the relevant code at places to produce
the dump in xml format.
mysql-test/r/mysqldump.result:
Added a test case for Bug#11760384.
mysql-test/t/mysqldump.test:
Added a test case for Bug#11760384.
--FLUSH-LOG BREAKS CONSISTENCY
The transaction started by mysqldump gets committed
implicitly when flush-log is specified along with
single-transaction option, and hence can break
consistency.
This is because, COM_REFRESH is executed in order
to flush logs and starting from 5.5 this command
performs an implicit commit.
Fixed by making sure that COM_REFRESH is executed
before the transaction has started and not after it.
Note : This patch triggers following behavioral
changes in mysqldump :
1) After this patch we no longer flush logs before
dumping each database if --single-transaction
option is given like it was done before (in the
absence of --lock-all-tables and --master-data
options).
2) Also, after this patch, we start acquiring
FTWRL before flushing logs in cases when only
--single-transaction and --flush-logs are given.
It becomes safe to use mysqldump with these two
options and without --master-data parameter for
backups.
client/mysqldump.c:
Bug#12809202 61854: MYSQLDUMP --SINGLE-TRANSACTION
--FLUSH-LOG BREAKS CONSISTENCY
Added logic to make sure that, if flush-log option
is specified, mysql_refresh() is never executed after
the transaction has started.
Added verbose messages for all the executions of
mysql_refresh() in order to track its invocation.
mysql-test/r/mysqldump.result:
Added test case for Bug#12809202.
mysql-test/t/mysqldump.test:
Added test case for Bug#12809202.
("-") IN DATABASE NAMES IN ALTER DATABASE.
mysqldump did not quote database name in 'ALTER DATABASE'
statements in its output. This can further cause a failure
while loading if database name contains a hyphen '-'.
This happened as, while printing the 'ALTER DATABASE'
statements, the database name was not quoted.
Fixed by quoting the database name.
client/mysqldump.c:
Bug#11766310 : 59398: MYSQLDUMP 5.1 CAN'T HANDLE A DASH
("-") IN DATABASE NAMES IN ALTER DATABASE.
Modified the print statement in order to print the quoted
database name for 'ALTER DATABASE' statements.
mysql-test/r/mysqldump.result:
Added a test case for bug#11766310.
mysql-test/t/mysqldump.test:
Added a test case for bug#11766310.
"set optimizer_switch to e or d causes invalid memory writes/valgrind warnings":
due to prefix support, the argument "e" was overwritten with its full value
"engine_condition_pushdown", which caused a buffer overrun.
This was wrong usage of find_type(); other wrong usages are fixed here too.
Please start reading with the comment of typelib.c.
client/mysqldump.c:
A bug: find_type() expects a bitmap as 3rd argument
(each bit is a flag controlling a behaviour of the function);
here it was instead passed the length of the string to search!
That could give random behaviour of find_type()
depending on the string.
We rather need to pass a correct flag to find_type().
The correct flag is FIND_TYPE_BASIC (0).
Flag 8 is not needed as buff cannot have a comma (see how buff is filled).
Flag 1 looks like a superfluous restriction.
Flag 4 is not user-friendly (why use
--compatible=2 rather than --compatible=mysql40 ?, and
we probably not commit to "2" always meaning "mysql40"
until the end of times).
include/mysql.h.pp:
This isn't a problematic API change as we go from char* to const char*:
existing code will run unchanged.
include/typelib.h:
named constants. Not an enum to not significantly change
the declaration of find_type() which would be an API change
(typelib.h is included in mysql.h).
mysql-test/r/mysqldump.result:
correct result (see the two requested modes in SQL_MODE)
mysql-test/suite/sys_vars/t/optimizer_switch_basic.test:
test for BUG#59894. The second SET used to crash.
mysql-test/t/mysqldump.test:
we had no test for multiple modes in --compatible, which is
supported according to --help
mysys/typelib.c:
Fix for BUG#59894. parse_name() is asked to match "e" with a row
of the TYPELIB (the TYPELIB lists permitted flags of optimizer_switch;
and comes from optimizer_switch_names[] of sys_vars.cc).
find_type() is capable of supporting prefixes, but if it is not
passed flag 2 in third argument, it will overwrite its first
argument (the string to search for) with the complete name,
here overwriting "e" with "engine_condition_pushdown". But
as this "e" was a buffer allocated in an Item, it was not big
enough to host the longer name, thus the crash.
We don't need to know the complete flag's name; the output used
from find_type() is just the flag's number (== function's return
code). So we can pass flag 2 to find_type() in parse_name().
After doing this fix and the other fixes in this patch, all usages
of find_type() were using flag 2; in most usages the string to search for,
is not guaranteed to be long enough to host the complete name
(it is either directly from argv, or from alloc_root/my_malloc
done in an earlier call).
Thus, flag 2 is here made implicit: callers need not pass it anymore,
it is always automatically turned on.
This allows to eliminate an oddity: parse_name() took a const char**,
and then removed "const" before calling find_type(), which could
theoretically modify the pointed data, thus lying on constness.
Last, constants for find_type() are now named.
sql-common/client.c:
Two bugs:
1) The enum was not in sync with the array (due to a bad porting of WL 1054;
the extra OPT_ values are about options present in 5.1 and deleted in 5.5);
added a compile_time_assert() to make sure this doesn't happen again
2) find_type() was writing past the end of opt_arg; as opt_arg was allocated
with alloc_root() with no extra space, this was an overrun; it could be seen
when
** building with -DWITH_VALGRIND -DHAVE_purify -DEXTRA_DEBUG
** making execution go through the faulty code; this faulty
code is executed only if the client asks to read a configuration
file like this:
mysql_options(mysql, MYSQL_READ_DEFAULT_FILE, "/tmp/cnf.cnf");
so by adding such line to the start of mysql_client_test.c::client_connect(),
we could see the valgrind warning:
==30548== Invalid write of size 1
==30548== at 0x4C2624C: strcpy (mc_replace_strmem.c:303)
==30548== by 0x48DC29: find_type (typelib.c:120)
==30548== by 0x465686: mysql_read_default_options (client.c:1344)
==30548== by 0x46830F: mysql_real_connect (client.c:2971)
==30548== by 0x409339: client_connect (mysql_client_test.c:331)
==30548== by 0x463A7F: main (mysql_client_test.c:19902)
==30548== Address 0x61875ad is 0 bytes after a block of size 29 alloc'd
==30548== at 0x4C25153: malloc (vg_replace_malloc.c:195)
==30548== by 0x49BFF1: my_malloc (my_malloc.c:38)
==30548== by 0x49C65C: alloc_root (my_alloc.c:166)
==30548== by 0x48EF97: handle_default_option (default.c:381)
==30548== by 0x49068C: search_default_file_with_ext (default.c:992)
==30548== by 0x48F929: search_default_file (default.c:670)
==30548== by 0x48EDC4: my_search_option_files (default.c:312)
==30548== by 0x48F4B1: my_load_defaults (default.c:576)
==30548== by 0x46517A: mysql_read_default_options (client.c:1207)
==30548== by 0x46830F: mysql_real_connect (client.c:2971)
==30548== by 0x409339: client_connect (mysql_client_test.c:331)
==30548== by 0x463A7F: main (mysql_client_test.c:19902)
This is fixed by having find_type() not overwrite anymore.
sql/sql_help.cc:
cast not needed anymore.
sql/table.cc:
cast not needed anymore.
When mysqldump tries to dump information in xml format,
the result does not contain field level comments.
In order to retrieve various informations for a field/column,
mysqldump currently uses 'show fields from <tab>' statement.
The attributes returned by the statement lacks the information
regarding field comments.
Fixed by changing the query to one that probes I_S to retrieve
required field informations, including the field comment.
client/mysqldump.c:
Bug#13618 : mysqldump --xml omits comment on table field.
Replaced the 'show fields' command by a statement that
queries I_S, in order to retrieve information on all the
attributes that 'show fields' returns along-with an additional
column_comment information.
mysql-test/r/client_xml.result:
Result modifications for bug#13618.
mysql-test/r/mysqldump.result:
Result modifications for bug#13618.
mysql-test/t/mysqldump.test:
Added a testcase for bug#13618.
Conflicts:
Text conflict in mysql-test/r/grant.result
Text conflict in mysql-test/t/grant.test
Text conflict in mysys/mf_loadpath.c
Text conflict in sql/slave.cc
Text conflict in sql/sql_priv.h
truncates text/blob to 766 chars
mysqldump and SELECT ... INTO OUTFILE truncated long BLOB/TEXT
values to size of 766 bytes (MAX_FIELD_WIDTH or 255 * 3 + 1).
The select_export::send_data method has been modified to
reallocate a conversion buffer for long field data.
mysql-test/r/mysqldump.result:
Test case for bug #53088.
mysql-test/r/outfile_loaddata.result:
Test case for bug #53088.
mysql-test/t/mysqldump.test:
Test case for bug #53088.
mysql-test/t/outfile_loaddata.test:
Test case for bug #53088.
sql/sql_class.cc:
Bug #53088: mysqldump with -T & --default-character-set set
truncates text/blob to 766 chars
The select_export::send_data method has been modified to
reallocate a conversion buffer for long field data.
The task is to
(a) add a comment on indexes and
(b) increase the maximum length of column, table and the new index comments.
The patch committed on behalf of Yoshinori Matsunobu (Yoshinori.Matsunobu@Sun.COM).
to string conversions and vice versa"
Initial import of the dtoa.c code and custom wrappers around it
to allow its usage from the server code.
Conversion of FLOAT/DOUBLE values to DECIMAL ones or strings
and vice versa has been significantly reworked. As the new
algoritms are more precise than the older ones, results of such
conversions may not always match those obtained from older
server versions. This in turn may break compatibility for some
applications.
This patch also fixes the following bugs:
- bug #12860 "Difference in zero padding of exponent between
Unix and Windows"
- bug #21497 "DOUBLE truncated to unusable value"
- bug #26788 "mysqld (debug) aborts when inserting specific
numbers into char fields"
- bug #24541 "Data truncated..." on decimal type columns
without any good reason"
Conflicts
=========
Text conflict in .bzr-mysql/default.conf
Text conflict in libmysqld/CMakeLists.txt
Text conflict in libmysqld/Makefile.am
Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/extra/rpl_tests/rpl_row_sp006.test
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata.result
Text conflict in mysql-test/suite/rpl/r/rpl_loaddata_fatal.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_create_table.result
Text conflict in mysql-test/suite/rpl/r/rpl_row_sp006_InnoDB.result
Text conflict in mysql-test/suite/rpl/r/rpl_stm_log.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_circular_simplex.result
Text conflict in mysql-test/suite/rpl_ndb/r/rpl_ndb_sp006.result
Text conflict in mysql-test/t/mysqlbinlog.test
Text conflict in sql/CMakeLists.txt
Text conflict in sql/Makefile.am
Text conflict in sql/log_event_old.cc
Text conflict in sql/rpl_rli.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_binlog.cc
Text conflict in sql/sql_lex.h
21 conflicts encountered.
NOTE
====
mysql-5.1-rpl-merge has been made a mirror of mysql-next-mr:
- "mysql-5.1-rpl-merge$ bzr pull ../mysql-next-mr"
This is the first cset (merge/...) committed after pulling
from mysql-next-mr.
InnoDB specified.
NO_ENGINE_SUBSTITUTION added to TRADITIONAL sql mode to prevent
silent conversions from InnoDB to MyISAM in that sql mode.
A number of test case results files updated to reflect this change.
Test added to sql_mode.test that checks that TRADITIONAL really
includes NO_ENGINE_SUBSTITUION.
mysql-test/t/ctype_utf8.test:
This test lacked "--source include/have_innodb.inc" which meant that
a number of DDL statements with engine=innodb in reality were using
myisam. "--disable_warnings" around all such statements, meant that
these engine substitutions were not visible. Test case has been
updated to include have_innodb.inc and "--disable_warnings" have been
removed for the relevant DDL statements.
Original revision in mysql-6.0-codebase is:
revno: 2617.31.14
committer: Konstantin Osipov <kostja@sun.com>
branch nick: mysql-6.0-runtime
timestamp: Sat 2009-03-28 11:42:55 +0300
message:
Bug#35297 SHOW CREATE EVENT does not show the DEFINER:
update test result after a merge from now.
DECIMAL and TIMESTAMP used to have NUM_FLAG, but NEWDECIMAL was forgotten.
It's correct that TIMESTAMP does not have the flag nowadays (manual will be updated, connectors
developers will be notified).
client/mysqldump.c:
IS_NUM_FIELD(f) removed and replaced by its definition (f>flags & NUM_FLAG).
include/mysql.h:
- IS_NUM_FIELD() is removed because name is too close to IS_NUM() and it is not used a lot
- INTERNAL_NUM_FIELD() is removed:
* it forgets to test NEWDECIMAL (when IS_NUM() was updated for NEWDECIMAL we forgot
to update INTERNAL_NUM_FIELD()), that's why client didn't mark NEWDECIMAL with NUM_FLAG (a bug).
* it has an obsolete test for length of the TIMESTAMP field: test became accidentally wrong when length
of TIMESTAMP was changed to always be 19 (when the format was changed from YYYYMMDDhhmmss to
YYYY-MM-DD hh:mm:ss), never 8 or 14 anymore. That obsolete test caused TIMESTAMP to lose NUM_FLAG,
which was an accidental but good change (see below).
* IS_NUM() should be used instead
- IS_NUM(f) is changed: TIMESTAMP used to be parsable as a number without quotes (when it was formatted as
"YYYYMMDDhhmmss"); but it is not anymore (now that it is "YYYY-MM-DD hh:mm:ss"), so it should not have NUM_FLAG
(mysqldump needs to quote TIMESTAMP values), so IS_NUM() should return false for it.
libmysqld/lib_sql.cc:
use IS_NUM() instead of INTERNAL_NUM_FIELD()
mysql-test/r/bigint.result:
result change: NEWDECIMAL fields now have NUM_FLAG (32768)
mysql-test/r/metadata.result:
result change: NEWDECIMAL fields now have NUM_FLAG (32768)
mysql-test/r/mysqldump.result:
DECIMAL columns are not quoted anymore by mysqldump. Which is ok, the parser does not need '' for them
mysql-test/r/ps_2myisam.result:
result change: NEWDECIMAL fields now have NUM_FLAG (32768)
mysql-test/r/ps_3innodb.result:
result change: NEWDECIMAL fields now have NUM_FLAG (32768)
mysql-test/r/ps_4heap.result:
result change: NEWDECIMAL fields now have NUM_FLAG (32768)
mysql-test/r/ps_5merge.result:
result change: NEWDECIMAL fields now have NUM_FLAG (32768)
mysql-test/suite/ndb/r/ps_7ndb.result:
result change: NEWDECIMAL fields now have NUM_FLAG (32768)
mysql-test/t/metadata.test:
test for BUG#42980
sql-common/client.c:
use IS_NUM() instead of INTERNAL_NUM_FIELD()
when used with --tab
1) New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE).
mysqldump is updated to use this in --tab mode.
2) ESCAPED BY/ENCLOSED BY field parameters are documented as
accepting CHAR argument, however SELECT .. INTO OUTFILE
silently ignored rests of multisymbol arguments.
For the symmetrical behavior with LOAD DATA INFILE the
server has been modified to fail with the same error:
ERROR 42000: Field separator argument is not what is
expected; check the manual
3) Current LOAD DATA INFILE recognizes field/line separators
"as is" without converting from client charset to data
file charset. So, it is supposed, that input file of
LOAD DATA INFILE consists of data in one charset and
separators in other charset. For the compatibility with
that [buggy] behaviour SELECT INTO OUTFILE implementation
has been saved "as is" too, but the new warning message
has been added:
Non-ASCII separator arguments are not fully supported
This message warns on field/line separators that contain
non-ASCII symbols.
client/mysqldump.c:
mysqldump has been updated to call SELECT ... INTO OUTFILE
statement with a charset from the --default-charset command
line parameter.
mysql-test/r/mysqldump.result:
Added test case for bug #30946.
mysql-test/r/outfile_loaddata.result:
Added test case for bug #30946.
mysql-test/t/mysqldump.test:
Added test case for bug #30946.
mysql-test/t/outfile_loaddata.test:
Added test case for bug #30946.
sql/field.cc:
String conversion code has been moved from check_string_copy_error()
to convert_to_printable() for reuse.
sql/share/errmsg.txt:
New WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED message has been added.
sql/sql_class.cc:
The select_export::prepare() method has been modified to:
1) raise the ER_WRONG_FIELD_TERMINATORS error on multisymbol
ENCLOSED BY/ESCAPED BY field arguments like LOAD DATA INFILE;
2) warn with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
message on non-ASCII field or line separators.
The select_export::send_data() merhod has been modified to
convert item data to output charset (see new SELECT INTO OUTFILE
syntax). By default the BINARY charset is used for backward
compatibility.
sql/sql_class.h:
The select_export::write_cs field added to keep output
charset.
sql/sql_load.cc:
mysql_load has been modified to warn on non-ASCII field or
line separators with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
message.
sql/sql_string.cc:
New global function has been added: convert_to_printable()
(common code has been moved from check_string_copy_error()).
sql/sql_string.h:
New String::is_ascii() method and new global convert_to_printable()
function have been added.
sql/sql_yacc.yy:
New syntax: added CHARACTER SET clause to the
SELECT ... INTO OUTFILE (to complement the same clause in
LOAD DATA INFILE). By default the BINARY charset is used for
backward compatibility.
mysqldump --tab still dumped triggers to stdout rather than to
individual tables.
We now append triggers to the .sql file for the corresponding
table.
--events and --routines correspond to a database rather than a
table and will still go to stdout with --tab unless redirected
with --result-file (-r).
client/mysqldump.c:
Extend open_sql_file_for_table() so we can open-append.
Change dump_triggers_for_table() so it will append its
output to the table's .sql-file when --tab is used.
mysql-test/r/mysqldump.result:
Show that when using --tab, triggers now end up in the .sql
file with the corresponding table (after that table), while
--routines and --events go to stdout or --result-file.
mysql-test/t/mysqldump.test:
Show that when using --tab, triggers now end up in the .sql
file with the corresponding table (after that table), while
--routines and --events go to stdout or --result-file.
mysqldump.test is designed to run with concurrent inserts
disabled. It is disabling concurrent inserts at the very
beginning of the test case, and re-enables them at the
bottom of the test. But for some reason (likely incorrect
merge) we enable concurrent inserts in the middle of the test.
The problem is fixed by enabling concurrent inserts only
at the bottom of the test case.
--ignore-table option
mysqldump would correctly omit temporary tables for views, but would
incorrectly still emit all CREATE VIEW statements.
Backport a fix from 5.1, where we capture the names we want to emit
views for in one pass (the placeholder tables) and in the pass where
we actually emit the views, we don't emit a view if it wasn't in that
list.
print compact view name if the view belongs to the current database
mysql-test/r/information_schema_db.result:
result fix
mysql-test/r/mysqldump.result:
result fix
mysql-test/r/view_grant.result:
result fix
sql/sql_show.cc:
print compact view name if the view belongs to the current database
mysqldump included character_set_client magic
that is unknown before 4.1 even when asked for
an appropriate compatibility mode.
In compatibility (3.23, 4.0) mode, we do not
output charset statements (not even in a
"comment conditional"), nor do we do magic on
the server, even if the server is sufficient
new (4.1+). Table-names will be output converted
to the charset requested by mysqldump; if such
a conversion is not possible (Ivrit -> Latin),
mysqldump will fail.
- If missing: add "disconnect <session>"
- If physical disconnect of non "default" sessions is not finished
at test end: add routine which waits till this happened
+ additional improvements like
- remove superfluous files created by the test
- replace error numbers by error names
- remove trailing spaces, replace tabs by spaces
- unify writing of bugs within comments
- correct comments
- minor changes of formatting
Modifications according to the code review are included.
Fixed tests:
grant2
grant3
lock_tables_lost_commit
mysqldump
openssl_1
outfile
mysqldump included character_set_client magic
that is unknown before 4.1 even when asked for
an appropriate compatibility mode.
In compatibility (3.23, 4.0) mode, we do not
output charset statements (not even in a
"comment conditional"), nor do we do magic on
the server, even if the server is sufficient
new (4.1+). Table-names will be output converted
to the charset requested by mysqldump; if such
a conversion is not possible (Ivrit -> Latin),
mysqldump will fail.
client/mysqldump.c:
in 3.23/4.0 compat mode, don't do charset magic,
period. not in output, but not on the server,
either!
mysql-test/r/mysqldump-max.result:
character_set_client magic lives in version-conditional
now (except in compat 3.23/4.0 mode, in which case we
don't output any at all!).
mysql-test/r/mysqldump.result:
character_set_client magic lives in version-conditional
now (except in compat 3.23/4.0 mode, in which case we
don't output any at all!).
mysql-test/r/openssl_1.result:
character_set_client magic lives in version-conditional
now (except in compat 3.23/4.0 mode, in which case we
don't output any at all!).
mysql-test/t/mysqldump.test:
character_set_client magic lives in version-conditional
now (except in compat 3.23/4.0 mode, in which case we
don't output any at all!).
Merge of fixes from 5.0 -> 5.1
Moved restoration of concurrent_insert's original value to the end of the 5.1 tests
Re-recorded .result file to account for changes to test file.
Moved fix for this bug to 5.0 as other mysqldump bugs seem tied to concurrent_insert being on
Setting concurrent_insert off during this test as INSERTs weren't being
completely processed before the calls to mysqldump, resulting in failing tests.
Altered .test file to turn concurrent_insert off during the test and to restore it
to whatever the value was at the start of the test when complete.
Re-recorded .result file to account for changes to variables in the test.
mysqldump creates stand-in tables before dumping the actual view.
Those tables were of the default type; if the view had more columns
than that (a pathological case, arguably), loading the dump would
fail. We now make the temporary stand-ins MyISAM tables to prevent
this.
client/mysqldump.c:
When creating a stand-in table, specify its type to
avoid defaulting to a type with a column-number limit
(like Inno). The type is always MyISAM as we know that
to be available.
mysql-test/r/mysqldump-max.result:
add test results for 31434
mysql-test/r/mysqldump.result:
mysqldump sets engine-type (MyISAM) for stand-in tables
for views now. Update test results.
mysql-test/t/mysqldump-max.test:
Show that mysqldump's stand-in tables for views explicitly
set engine-type to MyISAM to avoid falling back on an engine
that might support fewer columns than the final view requires
(here's lookin' at you, inno). Also show that this actually
has the desired effect by dumping and reloading a view that
has more columns than inno supports.
