ERROR MESSAGE
Description:- Mtr test, "main.mysqldump" is failing with an
assert when "mysqlimport" client utility is executed with
the option "--use_threads".
Analysis:- "mysqlimport" uses the option, "--use_threads",
to spawn worker threads to complete its job in parallel. But
currently the main thread is not waiting for the worker
threads to complete its cleanup, rather just wait for the
worker threads to say its done doing its job. So the cleanup
is done in a race between the worker threads and the main
thread. This lead to an assertion failure.
Fix:- "my_thread_join()" is introduced in the main thread to
join all the worker threads it have spawned. This will let
the main thread to wait for all the worker threads to
complete its cleanup before calling "my_end()".
IMPLEMENTED IN ALL CLIENT PROGRAMS
Description: Option "enable-cleartext-plugin" is not
available for the following client utilities:-
mysqldump
mysqlimport
mysqlshow
mysqlcheck
Analysis: The unavailability of this option limits the
features like PAM authentication from using the above
mentioned utilities.
Fix: Option "enable-cleartext-plugin" is implemented in the
above mentioned client utilities.
MULTIPLE THREADS
Description:- The utility "mysqlimport" does not use
multiple threads for the execution with option
"--use-threads". "mysqlimport" while importing multiple
files and multiple tables, uses a single thread even if the
number of threads are specified with "--use-threads" option.
Analysis:- This utility uses ifdef HAVE_LIBPTHREAD to check
for libpthread library and if defined uses libpthread
library for mutlithreaing. Since HAVE_LIBPTHREAD is not
defined anywhere in the source, "--use-threads" option is
silently ignored.
Fix:- "-DTHREADS" is set to the COMPILE_FLAGS which will
enable pthreads. HAVE_LIBPTHREAD macro is removed.
SHOW 2012 INSTEAD OF 2011
* Added a new macro to hold the current year :
COPYRIGHT_NOTICE_CURRENT_YEAR
* Modified ORACLE_WELCOME_COPYRIGHT_NOTICE macro
to take the initial year as parameter and pick
current year from the above mentioned macro.
TOOLS
Backport a fix for Bug 57094 from 5.5.
The following revision was backported:
# revision-id: alexander.nozdrin@oracle.com-20101006150613-ls60rb2tq5dpyb5c
# parent: bar@mysql.com-20101006121559-am1e05ykeicwnx48
# committer: Alexander Nozdrin <alexander.nozdrin@oracle.com>
# branch nick: mysql-5.5-bugteam-bug57094
# timestamp: Wed 2010-10-06 19:06:13 +0400
# message:
# Fix for Bug 57094 (Copyright notice incorrect?).
#
# The fix is to:
# - introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
# to specify copyright notice;
# - replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
# in programs.
The fix is to:
- introduce ORACLE_WELCOME_COPYRIGHT_NOTICE define to have a single place
to specify copyright notice;
- replace custom copyright notices with ORACLE_WELCOME_COPYRIGHT_NOTICE
in programs.
Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
strict aliasing violations.
Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.
As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.
The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.
Conflicts:
Text conflict in .bzr-mysql/default.conf
Text conflict in mysql-test/suite/rpl/r/rpl_slow_query_log.result
Text conflict in mysql-test/suite/rpl/t/rpl_slow_query_log.test
Conflict adding files to server-tools. Created directory.
Conflict because server-tools is not versioned, but has versioned children. Versioned directory.
Conflict adding files to server-tools/instance-manager. Created directory.
Conflict because server-tools/instance-manager is not versioned, but has versioned children. Versioned directory.
Contents conflict in server-tools/instance-manager/options.cc
Text conflict in sql/mysqld.cc
----------------------------------------------------------------------
ChangeSet@1.2571, 2008-04-08 12:30:06+02:00, vvaintroub@wva. +122 -0
Bug#32082 : definition of VOID in my_global.h conflicts with Windows
SDK headers
VOID macro is now removed. Its usage is replaced with void cast.
In some cases, where cast does not make much sense (pthread_*, printf,
hash_delete, my_seek), cast is ommited.
added code to exit a tool if the forced config file wasn't found
per-file comments:
client/mysql.cc
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysql_upgrade.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqladmin.cc
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlcheck.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqldump.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlimport.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlshow.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
client/mysqlslap.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
mysql-test/t/mysql.test
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
test added
sql/mysqld.cc
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
storage/myisam/myisamchk.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
storage/myisam/myisampack.c
Bug#47216 programs should quit if the file specified by --defaults-file option isn't found
added code to exit a tool if the forced config file wasn't found
Option "--without-server" still not working in 5.1
The general approach is to make sure that source files
which require thread support are only compiled if the build
really needs thread support,
which means when the server is built or a thread-safe client
library.
This required several changes:
- Make sure the subdirectories "storage/" and "plugin/" are
only processed if the server is built, not ifclient-only.
- Make the compilation of some modules which inherently
require threading depend on thread supportin the build.
- Separate the handling of threading in "configure.in" from
that of server issues, threading is also needed in a
non-server build of a thread-safe client library.
Also, "libdbug" must get built even in a client-only build,
so "dbug/" must be in the list of client directories.
In addition, calls to thread functions in source files which
can be built without thread support must use the wrapper
functions which handle the non-threaded build.
So the modules "client/mysqlimport.c" and "client/mysqlslap.c"
must call "my_thread_end()" only via "mysql_thread_end()".
make sure that if builder configured with a non-standard (!= 3306)
default TCP port that value actually gets used throughout. if they
didn't configure a value, assume "use a sensible default", which
will be read from /etc/services or, failing that, from the factory
default. That makes the order of preference
- command-line option
- my.cnf, where applicable
- $MYSQL_TCP_PORT environment variable
- /etc/services (unless configured --with-tcp-port)
- default port (--with-tcp-port=... or factory default)
Fixed bug in query cache that made it impossible to run mysqld with --debug
Fixed memory leaks in mysqldump and mysqltest
Memory leaks associated with wrong usage of mysqltest is not fixed. To find these, run
mysql-test-run --debug mysqltest
Fixed compiler warnings, errors and link errors
Fixed new bug on Solaris with gethrtime()
Added --debug-check option to all mysql clients to print errors and memory leaks
Added --debug-info to all clients. This now works as --debug-check but also prints memory and cpu usage
The following type conversions was done:
- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t
Removed declaration of byte, gptr, my_string, my_size_t and size_s.
Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
as this requires fewer casts in the code and is more in line with how the
standard functions work.
- Added extra length argument to dirname_part() to return the length of the
created string.
- Changed (at least) following functions to take uchar* as argument:
- db_dump()
- my_net_write()
- net_write_command()
- net_store_data()
- DBUG_DUMP()
- decimal2bin() & bin2decimal()
- Changed my_compress() and my_uncompress() to use size_t. Changed one
argument to my_uncompress() from a pointer to a value as we only return
one value (makes function easier to use).
- Changed type of 'pack_data' argument to packfrm() to avoid casts.
- Changed in readfrm() and writefrom(), ha_discover and handler::discover()
the type for argument 'frmdata' to uchar** to avoid casts.
- Changed most Field functions to use uchar* instead of char* (reduced a lot of
casts).
- Changed field->val_xxx(xxx, new_ptr) to take const pointers.
Other changes:
- Removed a lot of not needed casts
- Added a few new cast required by other changes
- Added some cast to my_multi_malloc() arguments for safety (as string lengths
needs to be uint, not size_t).
- Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done
explicitely as this conflict was often hided by casting the function to
hash_get_key).
- Changed some buffers to memory regions to uchar* to avoid casts.
- Changed some string lengths from uint to size_t.
- Changed field->ptr to be uchar* instead of char*. This allowed us to
get rid of a lot of casts.
- Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar
- Include zlib.h in some files as we needed declaration of crc32()
- Changed MY_FILE_ERROR to be (size_t) -1.
- Changed many variables to hold the result of my_read() / my_write() to be
size_t. This was needed to properly detect errors (which are
returned as (size_t) -1).
- Removed some very old VMS code
- Changed packfrm()/unpackfrm() to not be depending on uint size
(portability fix)
- Removed windows specific code to restore cursor position as this
causes slowdown on windows and we should not mix read() and pread()
calls anyway as this is not thread safe. Updated function comment to
reflect this. Changed function that depended on original behavior of
my_pwrite() to itself restore the cursor position (one such case).
- Added some missing checking of return value of malloc().
- Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow.
- Changed type of table_def::m_size from my_size_t to ulong to reflect that
m_size is the number of elements in the array, not a string/memory
length.
- Moved THD::max_row_length() to table.cc (as it's not depending on THD).
Inlined max_row_length_blob() into this function.
- More function comments
- Fixed some compiler warnings when compiled without partitions.
- Removed setting of LEX_STRING() arguments in declaration (portability fix).
- Some trivial indentation/variable name changes.
- Some trivial code simplifications:
- Replaced some calls to alloc_root + memcpy to use
strmake_root()/strdup_root().
- Changed some calls from memdup() to strmake() (Safety fix)
- Simpler loops in client-simple.c
around the original functions. These will ensure that error
message is always in unique form, reduce code and print the
right alternatives automatically in an error case.
Removed a lot of compiler warnings
Removed not used variables, functions and labels
Initialize some variables that could be used unitialized (fatal bugs)
%ll -> %l
mysqlimport.c declared the opt_use_threads variable as my_bool, but the corresponding option was defined as GET_UINT. The problem only occured on architectures with strict alignment rules.
Corrected spelling in copyright text
Makefile.am:
Don't update the files from BitKeeper
Many files:
Removed "MySQL Finland AB & TCX DataKonsult AB" from copyright header
Adjusted year(s) in copyright header
Many files:
Added GPL copyright text
Removed files:
Docs/Support/colspec-fix.pl
Docs/Support/docbook-fixup.pl
Docs/Support/docbook-prefix.pl
Docs/Support/docbook-split
Docs/Support/make-docbook
Docs/Support/make-makefile
Docs/Support/test-make-manual
Docs/Support/test-make-manual-de
Docs/Support/xwf
Moved .progress files into the log directory
Moved 'cluster' database tables into the MySQL database, to not have 'cluster' beeing a reserved database name
Fixed bug where mysqld got a core dump when trying to use a table created by MySQL 3.23
Fixed some compiler warnings
Fixed small memory leak in libmysql
Note that this doesn't changeset doesn't include the new mysqldump.c code required to run some tests. This will be added when I merge 5.0 to 5.1
