Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
------------------------------------------------------------
revno: 3317
revision-id: davi.arnaut@sun.com-20090522170916-fzc5ca3tjs9roy1t
parent: patrick.crews@sun.com-20090522152933-ole8s3suy4zqyvku
committer: Davi Arnaut <Davi.Arnaut@Sun.COM>
branch nick: 41860-6.0
timestamp: Fri 2009-05-22 14:09:16 -0300
message:
Bug#41860: Without Windows named pipe
The problem was that the patch for Bug#10374 broke named pipe
and shared memory transports on Windows due to a failure to
implement a dummy poll method for transports other than BSD
sockets. Another problem was that mysqltest lacked support
for named pipe and shared memory connections, which lead to
misleading test cases that were supposed run common queries
over both transports.
The solution is to properly implement, at the VIO layer, the
poll and is_connected methods. The is_connected method is
implemented for every suppported transport and the poll one
only where it makes sense. Furthermore, support for named pipe
and shared memory connections is added to mysqltest as to
enable testing of both transports using the test suite.
Bug#31621: Windows server hanging during shutdown using named pipes
and idle connection
Problem: when idle pipe connection is forcefully closed with KILL
statement or when the server goes down, thread that is closing connection
would hang infinitely in CloseHandle(). The reason for the hang is that
named pipe operations are performed synchronously. In this mode all IOs
on pipe are serialized, that is CloseHandle() will not abort ReadFile()
in another thread, but wait for ReadFile() to complete.
The fix implements asynchrnous mode for named pipes, where operation of file
are not synchronized. Read/Write operation would fire an async IO and wait for
either IO completion or timeout.
Note, that with this patch timeouts are properly handled for named pipes.
Post-review: Win32 timeout code has been fixed for named pipes and shared
memory. We do not store pointer to NET in vio structure, only the read and
write timeouts.
The following type conversions was done:
- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t
Removed declaration of byte, gptr, my_string, my_size_t and size_s.
Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
as this requires fewer casts in the code and is more in line with how the
standard functions work.
- Added extra length argument to dirname_part() to return the length of the
created string.
- Changed (at least) following functions to take uchar* as argument:
- db_dump()
- my_net_write()
- net_write_command()
- net_store_data()
- DBUG_DUMP()
- decimal2bin() & bin2decimal()
- Changed my_compress() and my_uncompress() to use size_t. Changed one
argument to my_uncompress() from a pointer to a value as we only return
one value (makes function easier to use).
- Changed type of 'pack_data' argument to packfrm() to avoid casts.
- Changed in readfrm() and writefrom(), ha_discover and handler::discover()
the type for argument 'frmdata' to uchar** to avoid casts.
- Changed most Field functions to use uchar* instead of char* (reduced a lot of
casts).
- Changed field->val_xxx(xxx, new_ptr) to take const pointers.
Other changes:
- Removed a lot of not needed casts
- Added a few new cast required by other changes
- Added some cast to my_multi_malloc() arguments for safety (as string lengths
needs to be uint, not size_t).
- Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done
explicitely as this conflict was often hided by casting the function to
hash_get_key).
- Changed some buffers to memory regions to uchar* to avoid casts.
- Changed some string lengths from uint to size_t.
- Changed field->ptr to be uchar* instead of char*. This allowed us to
get rid of a lot of casts.
- Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar
- Include zlib.h in some files as we needed declaration of crc32()
- Changed MY_FILE_ERROR to be (size_t) -1.
- Changed many variables to hold the result of my_read() / my_write() to be
size_t. This was needed to properly detect errors (which are
returned as (size_t) -1).
- Removed some very old VMS code
- Changed packfrm()/unpackfrm() to not be depending on uint size
(portability fix)
- Removed windows specific code to restore cursor position as this
causes slowdown on windows and we should not mix read() and pread()
calls anyway as this is not thread safe. Updated function comment to
reflect this. Changed function that depended on original behavior of
my_pwrite() to itself restore the cursor position (one such case).
- Added some missing checking of return value of malloc().
- Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow.
- Changed type of table_def::m_size from my_size_t to ulong to reflect that
m_size is the number of elements in the array, not a string/memory
length.
- Moved THD::max_row_length() to table.cc (as it's not depending on THD).
Inlined max_row_length_blob() into this function.
- More function comments
- Fixed some compiler warnings when compiled without partitions.
- Removed setting of LEX_STRING() arguments in declaration (portability fix).
- Some trivial indentation/variable name changes.
- Some trivial code simplifications:
- Replaced some calls to alloc_root + memcpy to use
strmake_root()/strdup_root().
- Changed some calls from memdup() to strmake() (Safety fix)
- Simpler loops in client-simple.c
- It's too early to free the SSL object in 'vio_ssl_close'. There
might still be a thread using or reading from it on platforms
where we need to close the active connection/socket in order
to break the read.
- Add new function 'vio_ssl_delete' and install it as the viodelete
function for SSL connections.
The patch implements the idea suggested by Olaf van der Spek in
thread "Client: many small reads?" (internals@lists.mysql.com).
Now small reads performed by the client library are buffered.
The buffering gives up to 2 times speedup when retrieving
one-column tables.
Add support for VARCHAR with 1 or 2 length bytes
Enable VARCHAR packing in MyISAM files (previous patch didn't pack data properly)
Give error if we got problems in temporary tables during a SELECT
Don't use new table generated by ALTER TABLE if index generation fails
Fixed wrong call by range_end() (Could cause an ASSERT in debug mode)
(continue)
added event_conn_closed
replaced WaitForSingleObject on WaitForMultipleObjects
inserted a check in vio_close()
added SetEvent() for event_conn_closed
Signed auto_increment keys for HASH tables (like for MyISAM tables in 4.0)
nitialize system_charset_info() early. Fixes core dump when starting windows service
His explanation:
The socket on which MySQL listens for new connections on a blocking
socket most of the time but is set to non-blocking during the
accept() of the new connection. Due to a bug in the kernel, the new
socket returned by accept() is a blocking socket but returns the
O_NONBLOCK flag when queried via fcntl(F_GETFL). That is, the file
descriptor and the underlying socket don't agree on the blocking
mode.
Since MySQL determines via fcntl(F_GETFL) that the socket is
non-blocking, it expects the first read() in my_real_read to not
block, so it doesn't enable the timeout alarm. However, the read
does block, and thus there's no timeout alarm. The thread kill
(which relies on rescheduling the timeout alarm) also does not work
as a consequence.
The bug shows itself if you build MySQL with LinuxThreads support
(needed for SMP on FreeBSD). Issuing a KILL command in MySQL won't be
"noticed" by the "killed" thread until it runs another query--that
makes KILL pretty useless. And the wait_timeout doesn't work either.
Fixed bugs in my last changeset that made MySQL hard to compile.
Added mutex around some data that could cause table cache corruptions when using OPTIMIZE TABLE / REPAIR TABLE or automatic repair of MyISAM tables.
Added mutex around some data in the slave start/stop code that could cause THD linked list corruptions
Extended my_chsize() to allow one to specify a filler character.
Extend vio_blocking to return the old state (This made some usage of this function much simpler)
Added testing for some functions that they caller have got the required mutexes before calling the function.
Use setrlimit() to ensure that we can write core file if one specifies --core-file.
Added --slave-compressed-protocol
Made 2 the minimum length for ft_min_word_len
Added variables foreign_key_checks & unique_checks.
Less logging from replication code (if not started with --log-warnings)
Changed that SHOW INNODB STATUS requre the SUPER privilege
More DBUG statements and a lot of new code comments
This wrapper noew will include all the necessary, system specific files,
which makes all normal source files much easier to write and maintain.
Portability fixes.
Fixed error number handling bug in mysqltest.
Fixed that error number from insert delayed is reported correctly.
merged new vio code with old violite code.
