- BUG#11986: Stored routines and triggers can fail if the code
has a non-ascii symbol
- BUG#16291: mysqldump corrupts string-constants with non-ascii-chars
- BUG#19443: INFORMATION_SCHEMA does not support charsets properly
- BUG#21249: Character set of SP-var can be ignored
- BUG#25212: Character set of string constant is ignored (stored routines)
- BUG#25221: Character set of string constant is ignored (triggers)
There were a few general problems that caused these bugs:
1. Character set information of the original (definition) query for views,
triggers, stored routines and events was lost.
2. mysqldump output query in client character set, which can be
inappropriate to encode definition-query.
3. INFORMATION_SCHEMA used strings with mixed encodings to display object
definition;
1. No query-definition-character set.
In order to compile query into execution code, some extra data (such as
environment variables or the database character set) is used. The problem
here was that this context was not preserved. So, on the next load it can
differ from the original one, thus the result will be different.
The context contains the following data:
- client character set;
- connection collation (character set and collation);
- collation of the owner database;
The fix is to store this context and use it each time we parse (compile)
and execute the object (stored routine, trigger, ...).
2. Wrong mysqldump-output.
The original query can contain several encodings (by means of character set
introducers). The problem here was that we tried to convert original query
to the mysqldump-client character set.
Moreover, we stored queries in different character sets for different
objects (views, for one, used UTF8, triggers used original character set).
The solution is
- to store definition queries in the original character set;
- to change SHOW CREATE statement to output definition query in the
binary character set (i.e. without any conversion);
- introduce SHOW CREATE TRIGGER statement;
- to dump special statements to switch the context to the original one
before dumping and restore it afterwards.
Note, in order to preserve the database collation at the creation time,
additional ALTER DATABASE might be used (to temporary switch the database
collation back to the original value). In this case, ALTER DATABASE
privilege will be required. This is a backward-incompatible change.
3. INFORMATION_SCHEMA showed non-UTF8 strings
The fix is to generate UTF8-query during the parsing, store it in the object
and show it in the INFORMATION_SCHEMA.
Basically, the idea is to create a copy of the original query convert it to
UTF8. Character set introducers are removed and all text literals are
converted to UTF8.
This UTF8 query is intended to provide user-readable output. It must not be
used to recreate the object. Specialized SHOW CREATE statements should be
used for this.
The reason for this limitation is the following: the original query can
contain symbols from several character sets (by means of character set
introducers).
Example:
- original query:
CREATE VIEW v1 AS SELECT _cp1251 'Hello' AS c1;
- UTF8 query (for INFORMATION_SCHEMA):
CREATE VIEW v1 AS SELECT 'Hello' AS c1;
fixed by using flush logs, dumping, then doing PURGE MASTER LOGS TO 'binfile', instead
of deleting the log files at the beginning.
NOTE: previously the delete-master-logs would reset the log names back to
filename.00001. Now the trailing number doesn't get reset. This may need to be
documented.
sometimes `mysqldump --hex-blob' overruned output buffer by '\0' byte.
The dump_table() function has been fixed to reserve 1 byte more for the
last '\0' byte of dumped string.
mysqldump generated output that set OLD_SQL_MODE twice, to different values
(for triggers), or not at all (for routines) in some cases.
---
Merge blasphemy.mysql.com:/home/tnurnberg/28223/50-28223
into blasphemy.mysql.com:/home/tnurnberg/28223/51-28223
The following type conversions was done:
- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t
Removed declaration of byte, gptr, my_string, my_size_t and size_s.
Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
as this requires fewer casts in the code and is more in line with how the
standard functions work.
- Added extra length argument to dirname_part() to return the length of the
created string.
- Changed (at least) following functions to take uchar* as argument:
- db_dump()
- my_net_write()
- net_write_command()
- net_store_data()
- DBUG_DUMP()
- decimal2bin() & bin2decimal()
- Changed my_compress() and my_uncompress() to use size_t. Changed one
argument to my_uncompress() from a pointer to a value as we only return
one value (makes function easier to use).
- Changed type of 'pack_data' argument to packfrm() to avoid casts.
- Changed in readfrm() and writefrom(), ha_discover and handler::discover()
the type for argument 'frmdata' to uchar** to avoid casts.
- Changed most Field functions to use uchar* instead of char* (reduced a lot of
casts).
- Changed field->val_xxx(xxx, new_ptr) to take const pointers.
Other changes:
- Removed a lot of not needed casts
- Added a few new cast required by other changes
- Added some cast to my_multi_malloc() arguments for safety (as string lengths
needs to be uint, not size_t).
- Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done
explicitely as this conflict was often hided by casting the function to
hash_get_key).
- Changed some buffers to memory regions to uchar* to avoid casts.
- Changed some string lengths from uint to size_t.
- Changed field->ptr to be uchar* instead of char*. This allowed us to
get rid of a lot of casts.
- Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar
- Include zlib.h in some files as we needed declaration of crc32()
- Changed MY_FILE_ERROR to be (size_t) -1.
- Changed many variables to hold the result of my_read() / my_write() to be
size_t. This was needed to properly detect errors (which are
returned as (size_t) -1).
- Removed some very old VMS code
- Changed packfrm()/unpackfrm() to not be depending on uint size
(portability fix)
- Removed windows specific code to restore cursor position as this
causes slowdown on windows and we should not mix read() and pread()
calls anyway as this is not thread safe. Updated function comment to
reflect this. Changed function that depended on original behavior of
my_pwrite() to itself restore the cursor position (one such case).
- Added some missing checking of return value of malloc().
- Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow.
- Changed type of table_def::m_size from my_size_t to ulong to reflect that
m_size is the number of elements in the array, not a string/memory
length.
- Moved THD::max_row_length() to table.cc (as it's not depending on THD).
Inlined max_row_length_blob() into this function.
- More function comments
- Fixed some compiler warnings when compiled without partitions.
- Removed setting of LEX_STRING() arguments in declaration (portability fix).
- Some trivial indentation/variable name changes.
- Some trivial code simplifications:
- Replaced some calls to alloc_root + memcpy to use
strmake_root()/strdup_root().
- Changed some calls from memdup() to strmake() (Safety fix)
- Simpler loops in client-simple.c
mysqldump didn't properly handle getting no data on
SHOW CREATE PROCEDURE. If S/C/P fails (due to dumping
user's insufficient privileges on mysql.proc, say),
mysqldump will print a comment to that effect to the
output and return an error-code. If the -f (force) option
is used, the dump will continue, otherwise, it will abort
right there and then.
Also fixes Bug#22761, "mysqldump reports no errors when using
--routines without mysql.proc privileges"
---
Merge mysql.com:/home/tnurnberg/27293/50-27293
into mysql.com:/home/tnurnberg/27293/51-27293
---
Merge tnurnberg@bk-internal.mysql.com:/home/bk/mysql-5.1-maint
into mysql.com:/home/tnurnberg/27293/51-27293
mysqldump didn't properly handle getting no data on
SHOW CREATE PROCEDURE. If S/C/P fails (due to dumping
user's insufficient privileges on mysql.proc, say),
mysqldump will print a comment to that effect to the
output and return an error-code. If the -f (force) option
is used, the dump will continue, otherwise, it will abort
right there and then.
Also fixes Bug#22761, "mysqldump reports no errors when using
--routines without mysql.proc privileges"
---
Merge tnurnberg@bk-internal.mysql.com:/home/bk/mysql-5.0-maint
into mysql.com:/home/tnurnberg/27293/50-27293
- Giving the directive '--all-databases' to mysqldump caused
an attempt to lock and dump log tables which don't support this
operation.
- With this patch the log tables are excluded from the set of
databases tables to dump.
around the original functions. These will ensure that error
message is always in unique form, reduce code and print the
right alternatives automatically in an error case.
BUG#26429: SHOW CREATE EVENT is incorrect for an event that
STARTS NOW()
BUG#26431: Impossible to re-create an event from backup if its
STARTS clause is in the past
WL#3698: Events: execution in local time zone
The problem was that local times specified by the user in AT, STARTS
and ENDS of CREATE EVENT/ALTER EVENT statement were converted to UTC,
and the original time zone was forgotten. This way, event scheduler
couldn't honor Daylight Saving Time shifts, and times shown to the
user were also in UTC. Additionally, CREATE EVENT didn't allow times
in the past, thus preventing straightforward event restoration from
old backups.
This patch reworks event scheduler time computations, performing them
in the time zone associated with the event. Also it allows times to
be in the past.
The patch adds time_zone column to mysql.event table.
NOTE: The patch is almost final, but the bug#9953 should be pushed
first.
Fixed compile-pentium64 scripts
Fixed wrong estimate of update_with_key_prefix in sql-bench
Merge bk-internal.mysql.com:/home/bk/mysql-5.1 into mysql.com:/home/my/mysql-5.1
Fixed unsafe define of uint4korr()
Fixed that --extern works with mysql-test-run.pl
Small trivial cleanups
This also fixes a bug in counting number of rows that are updated when we have many simultanous queries
Move all connection handling and command exectuion main loop from sql_parse.cc to sql_connection.cc
Split handle_one_connection() into reusable sub functions.
Split create_new_thread() into reusable sub functions.
Added thread_scheduler; Preliminary interface code for future thread_handling code.
Use 'my_thread_id' for internal thread id's
Make thr_alarm_kill() to depend on thread_id instead of thread
Make thr_abort_locks_for_thread() depend on thread_id instead of thread
In store_globals(), set my_thread_var->id to be thd->thread_id.
Use my_thread_var->id as basis for my_thread_name()
The above changes makes the connection we have between THD and threads more soft.
Added a lot of DBUG_PRINT() and DBUG_ASSERT() functions
Fixed compiler warnings
Fixed core dumps when running with --debug
Removed setting of signal masks (was never used)
Made event code call pthread_exit() (portability fix)
Fixed that event code doesn't call DBUG_xxx functions before my_thread_init() is called.
Made handling of thread_id and thd->variables.pseudo_thread_id uniform.
Removed one common 'not freed memory' warning from mysqltest
Fixed a couple of usage of not initialized warnings (unlikely cases)
Suppress compiler warnings from bdb and (for the moment) warnings from ndb
Removed a lot of compiler warnings
Removed not used variables, functions and labels
Initialize some variables that could be used unitialized (fatal bugs)
%ll -> %l
- Removed not used variables and functions
- Added #ifdef around code that is not used
- Renamed variables and functions to avoid conflicts
- Removed some not used arguments
Fixed some class/struct warnings in ndb
Added define IS_LONGDATA() to simplify code in libmysql.c
I did run gcov on the changes and added 'purecov' comments on almost all lines that was not just variable name changes
Fixed compiler warnings (detected by VC++):
- Removed not used variables
- Added casts
- Fixed wrong assignments to bool
- Fixed wrong calls with bool arguments
- Added missing argument to store(longlong), which caused wrong store method to be called.
- Removed not used variables
- Changed some ulong parameters/variables to ulonglong (possible serious bug)
- Added casts to get rid of safe assignment from longlong to long (and similar)
- Added casts to function parameters
- Fixed signed/unsigned compares
- Added some constructores to structures
- Removed some not portable constructs
Better fix for bug Bug #21428 "skipped 9 bytes from file: socket (3)" on "mysqladmin shutdown"
(Added new parameter to net_clear() to define when we want the communication buffer to be emptied)
Added missing DBUG_RETURN statements (in mysqldump.c)
Added missing enums
Fixed a lot of wrong DBUG_PRINT() statements, some of which could cause crashes
Removed usage of %lld and %p in printf strings as these are not portable or produces different results on different systems.
(Mostly in DBUG_PRINT() and unused arguments)
Fixed bug in query cache when used with traceing (--with-debug)
Fixed memory leak in mysqldump
Removed warnings from mysqltest scripts (replaced -- with #)
The mysqldump command with both the --xml and --hex-blob options will output blob data encoded as hexBinary.
The proper XML datatype is xs:hexBinary.
The correct XML datatype is specified be setting the xsi_type attribute equal to xs:hexBinary for each encoded element.
post-review fixes.
Magnus suggested use of DYNAMIC_STRING instead of futzing with c strings.
also making usage of TABLEs clearer in store_schema_partitions_record.
Update mysqldump to dump the needed tablespaces to create the tables
for either the dbs or tables we're dumping.
With --all-tablespaces, we still dump everything.
Dumps are created for the tables in each specified database then for the views in each specified database. This bug occurs when any database's views depend on the mysql database's table data while being restored.
Added command line option --flush-privileges to the mysqldump utility which causes a FLUSH PRIVILIGES statement to be written to the dump after the mysql database.
init_dumping now accepts a function pointer to the table or view specific init_dumping function. This allows both tables and views to use the init_dumping function.
The problem was that the error handling was using a too-small buffer to
print the error message generated. We fix this by not using a buffer at
all, but by using fprintf() directly. There were also some problems with
the error handling in table dumping that was exposed by this fix that were
also corrected.
mysqldump did not select the correct database before trying to dump
views from it. this resulted in an empty result set, which in turn
startled mysql-dump into a core-dump. this only happened for views,
not for tables, and was only visible with multiple databases that
weren't by sheer luck in the order mysqldump required, anyway. this
fixes by selecting the correct database before dumping views; it also
catches the empty set-condition if it should occur for other reasons.
mysqldump did not select the correct database before trying to dump
views from it. this resulted in an empty result set, which in turn
startled mysql-dump into a core-dump. this only happened for views,
not for tables, and was only visible with multiple databases that
weren't by sheer luck in the order mysqldump required, anyway. this
fixes by selecting the correct database before dumping views; it also
catches the empty set-condition if it should occur for other reasons.
(The above problem only occurs with -T -- create a separate file for
each table / view.) This ChangeSet results in correct output of view-
information while omitting the information for the view's stand-in
table. The rationale is that with -T, the user is likely interested
in transferring part of a database, not the db in its entirety (that
would be difficult as replay order is obscure, the files being named
for the table/view they contain rather than getting a sequence number).
'show create' works even on views that are short of a base-table (this
throw a warning though, like you would expect). Unfortunately, this is
not what mysqldump uses; it creates stand-in tables and hence requests
'show fields' on the view which fails with missing base-tables. The
--force option prevents the dump from stopping at this point; furthermore
this patch dumps a comment showing create for the offending view for
better diagnostics. This solution was confirmed by submitter as solving
their/clients' problem. Problem might become non-issue once mysqldump no
longer creates stand-in tables.
- Add new function 'ssl_verify_server_cert' which is used if we are
connecting to the server with SSL. It will compare the hostname in
the server's cert against the hostname that we used when connecting
to the server. Will reject the connection if hostname does not match.
- Add new option "OPT_SSL_VERIFY_SERVER_CERT" to be passed to mysql_options
which will turn on checking of servers cert.
- Add new argument "ssl-verify-server-cert" to all mysql* clients which
will activate the above option.
- Generate a new server cert with 1024 bits that has "localhost" as the server name.
The idea is to add DEFINER-clause in CREATE PROCEDURE and CREATE FUNCTION
statements. Almost all support of definer in stored routines had been already
done before this patch.
NOTE: this patch changes behaviour of dumping stored routines in mysqldump.
Before this patch, mysqldump did not dump DEFINER-clause for stored routines
and this was documented behaviour. In order to get full information about stored
routines, one should have dumped mysql.proc table. This patch changes this
behaviour, so that DEFINER-clause is dumped.
Since DEFINER-clause is not supported in CREATE PROCEDURE | FUNCTION statements
before this patch, the clause is covered by additional version-specific comments.
There are two main idea of this fix:
- introduce a common function for server and client to split user value
(<user name>@<host name>) into user name and host name parts;
- dump DEFINER clause in correct format in mysqldump.
So, use --replace and instead of getting INSERT INTO you will INSTEAD get REPLACE INTO. Buyer beward, REPLACE is a MySQL SQL, so you will not end up with a dump that can be used for other databases.
Though I hear you could just use a sed line to modify it back :)
mysqldump.result:
BUG# 12838
New test results for mysqldump -x on a DB with views
mysqldump.test:
sqldump.test:
BUG# 12838
New test to run mysqldump -x on a DB with views
mysqldump.c:
BUG# 12838
Removed/Changed code which created tables to be put into the dump
(For loading views of views) by creating temp tables and then using
the CREATE TABLE information in those temp tables. The problem with this
is that when mysqldump -x is called, it locks all tables, so the
temp tables could not be created, causing the mysqldump to exit with
failure. The code was changed to use SHOW FIELDS to get the column
names and type to build CREATE TABLE text used to create these tables
that views need in the dump.
Clean application of patch -
- Added --tz-utc to fix issue of dumping timestamp values between
servers with different global time zone settings, particularly
with regard to the day of DST changeover, which without this fix,
would dump duplicate timestamp values.
- CHAR() now returns binary string as default
- CHAR(X*65536+Y*256+Z) is now equal to CHAR(X,Y,Z) independent of the character set for CHAR()
- Test for both ETIMEDOUT and ETIME from pthread_cond_timedwait()
(Some old systems returns ETIME and it's safer to test for both values
than to try to write a wrapper for each old system)
- Fixed new introduced bug in NOT BETWEEN X and X
- Ensure we call commit_by_xid or rollback_by_xid for all engines, even if one engine has failed
- Use octet2hex() for all conversion of string to hex
- Simplify and optimize code
