Essentially, the problem is that safemalloc is excruciatingly
slow as it checks all allocated blocks for overrun at each
memory management primitive, yielding a almost exponential
slowdown for the memory management functions (malloc, realloc,
free). The overrun check basically consists of verifying some
bytes of a block for certain magic keys, which catches some
simple forms of overrun. Another minor problem is violation
of aliasing rules and that its own internal list of blocks
is prone to corruption.
Another issue with safemalloc is rather the maintenance cost
as the tool has a significant impact on the server code.
Given the magnitude of memory debuggers available nowadays,
especially those that are provided with the platform malloc
implementation, maintenance of a in-house and largely obsolete
memory debugger becomes a burden that is not worth the effort
due to its slowness and lack of support for detecting more
common forms of heap corruption.
Since there are third-party tools that can provide the same
functionality at a lower or comparable performance cost, the
solution is to simply remove safemalloc. Third-party tools
can provide the same functionality at a lower or comparable
performance cost.
The removal of safemalloc also allows a simplification of the
malloc wrappers, removing quite a bit of kludge: redefinition
of my_malloc, my_free and the removal of the unused second
argument of my_free. Since free() always check whether the
supplied pointer is null, redudant checks are also removed.
Also, this patch adds unit testing for my_malloc and moves
my_realloc implementation into the same file as the other
memory allocation primitives.
client/mysqldump.c:
Pass my_free directly as its signature is compatible with the
callback type -- which wasn't the case for free_table_ent.
Due to a BZR bug, that merge was done by the following command:
bzr merge -r 'revid:tor.didriksen@sun.com-20100527074248-6qtv0p1ugy6o1hjo..' <mysql-trunk-bugfixing path>
TO DO: Enable this in CMake-based builds.
------------------------------------------------------------
revno: 3474
revision-id: marko.makela@oracle.com-20100520104042-ma2nsscqdvwoph8k
parent: marko.makela@oracle.com-20100519081618-h38q02qxuvcowbtk
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: 5.1-innodb
timestamp: Thu 2010-05-20 13:40:42 +0300
message:
Bug#53593: Add some instrumentation to improve Valgrind sensitivity
BUILD/*: Add valgrind_configs=--with-valgrind.
BUILD/*: Remove -USAFEMALLOC from valgrind_flags.
configure.in: Add AC_ARG_WITH(valgrind) and HAVE_VALGRIND.
include/my_sys.h: Define a number of MEM_ wrappers for VALGRIND_ functions.
include/my_sys.h: Make TRASH do MEM_UNDEFINED().
include/m_string.h: Remove unused macro bzero_if_purify(A,B).
_mymalloc(): Declare MEM_UNDEFINED() on the allocated memory.
_myfree(): Declare MEM_NOACCESS() on the freed memory.
storage/innobase/include/univ.i: Enable UNIV_DEBUG_VALGRIND based on
HAVE_VALGRIND rather than HAVE_purify.
Possible things to do:
* In my_global.h, remove the defined(HAVE_purify) condition
from the _WIN32 uint3korr().
* In my_global.h *int*korr(), use | instead of +
in order to keep the Valgrind V bits accurate
* Consider replacing HAVE_purify with HAVE_VALGRIND
* Use VALGRIND_CREATE_BLOCK, VALGRIND_DISCARD in mem_root and similar places
(make relies GNU extentions). The patch was partially
backport from 6.0.
Original comment:
bug#30708: make relies GNU extensions. Now that we no longer use
BitKeeper we can safely remove the SCCS handling with no loss of
functionality.
The "cmake" way still shows issues in our release build environment.
Block it temporarily, but in a way that can easily be undone.
This change is to be reverted once the problems are solved.
BUILD/choose_configure.sh:
Easiest way: Preset "HAVE_CMAKE=no"
Reason for the error was that ./configure wrapper script was not
safe for VPATH builds used by "make distcheck", specifically it expected configure.am to be in the current directory.
Fix changes configure wrapper to resolve script names relative
to configure script path.
Also, use $top_srcdir/configure.am instead of configure.am in dist-hook to be more VPATH friendly.
The bug was that ./configure was passing paramers to subscripts as $@, and to handle embedded spaces it needs
to be quoted as "$@".
This resulting into a bug when ./configure was called e.g with CFLAGS='-m64 -Xstrconst'..
Additionally, fixed cmake/configure.pl did not handle environment variables passed on the command line.
this is fixed in this push
BUILD-CMAKE:
WL#5161 : Documentation on how to build with CMake on Unix/Windows
BUILD/Makefile.am:
Add new file
BUILD/autorun.sh:
WL#5161 : use choose_configure instead of autotools configure script
(choose configure will call cmake if cmake is available)
BUILD/choose_configure.sh:
WL#5161 : use choose_configure instead of autotools configure script
(choose configure will call cmake if cmake is available)
CMakeLists.txt:
WL#5161 : Rewrite top-level CMakeLists.txt.
Remove Windows specifics
- compiler flags handling moved to configure.cmake
- storage engine/plugin stuff moved into cmake/plugin.cmake
- copy docs
Makefile.am:
Added new files
client/CMakeLists.txt:
WL#5161 : Rewrite CMakeLists.txt to be platform-independent
Handle packagng (add INSTALL commands)
cmake/Makefile.am:
WL#5161 : use choose_configure instead of autotools configure script
(choose configure will call cmake if cmake is available)
cmake/abi_check.cmake:
Custom targets for abi_check (for cmake)
cmake/bison.cmake:
- Check bison availability
- Add RUN_BISON macro (used to create sql_yacc.cc and sql_yacc.h)
cmake/cat.cmake:
Add helper script to concatenate files.
cmake/character_sets.cmake:
Handle configuration parameters WITH_EXTRA_CHARSETS
cmake/check_minimal_version.cmake:
Helper script to check the minimum required version of cmake
cmake/configure.pl:
Add perl script to convert ./configure parameters for cmake
cmake/create_initial_db.cmake.in:
Add script helper to create initial database.
(on Windows, we pack initial db with the redistribution
package)
cmake/do_abi_check.cmake:
Perform abi check
cmake/dtrace.cmake:
Handle dtrace in CMake Build.
Check for dtrace availablility,
run dtrace -G on solaris in prelink step
cmake/dtrace_prelink.cmake:
Run dtrace -G on Solaris in pre-link step,
link the object it creates together with library or
executable
cmake/install_macros.cmake:
Helper macros for packaging
(install pdb on Windows, install symlinks on Unix)
cmake/make_dist.cmake.in:
"make dist" -
- pack autotools ./configure script with the source
(renamed to configure.am)
- pack bison output
cmake/merge_archives_unix.cmake.in:
script to merge static libraries on Unix
cmake/misc.cmake:
Build helper macros
- MERGE_STATIC_LIBS
We use it when building client library and embedded
(avoid recompilation)
- Convert source file paths to absolute names.
We use it in to locate files of a different project,
when the files need to be recompiled (e.g in embedded
several storage engines are recompiled with
-DEMBEDDED_LIBRARY)
cmake/mysql_version.cmake:
Extract version info from configure.in
Handle package names.
cmake/plugin.cmake:
Rewrote storage/mysql_storage_engine.cmake to handle
other types of plugins and do it in OS-independent manner.
cmake/readline.cmake:
Macros to handle WITH_READLINE/WITH_LIBEDIT parameters
cmake/ssl.cmake:
Add macros to handle WITH_SSL parameter.
cmake/stack_direction.c:
Helper to check stack direction.
cmake/zlib.cmake:
Add macros to handle WITH_ZLIB parameter
cmd-line-utils/libedit/CMakeLists.txt:
Build libedit with cmake.
cmd-line-utils/libedit/Makefile.am:
Add new file
cmd-line-utils/readline/CMakeLists.txt:
Build readline with CMake.
cmd-line-utils/readline/Makefile.am:
Add new file
config.h.cmake:
WL#5161 : Add config.h template for cmake
configure.cmake:
WL#5161 : Add platform tests ( for cmake)
configure.in:
Added new subdirectories
dbug/CMakeLists.txt:
WL#5161
extra/CMakeLists.txt:
WL#5161
extra/yassl/CMakeLists.txt:
WL#5161
extra/yassl/taocrypt/CMakeLists.txt:
WL#5161
include/Makefile.am:
Add new file
include/keycache.h:
remove configure-win.h and remove HUGE_PTR defined there.
include/my_global.h:
use my_config.h for Windows, not config-win.h anymore
include/my_pthread.h:
- Move thread_safe_increment from config-win.h to other headers
(config-win.h is not used anymore)
- Declare pthread_cancel on Windows (it is used in daemon_example)
include/my_sys.h:
Add malloc.h on Windows (we use -D_WIN32_LEAN_AND_MEAN now, and
with this define malloc.h is not included automatically via windows.h)
include/mysql/plugin.h:
Handle pure-C plugins with Microsoft compiler.
include/thr_alarm.h:
remove rf_SetTimer that used to be defined in config-win.h
Replace with UINT_PTR (we do not use config-win.h anymore
and typedef was needed in this single place only)
libmysql/CMakeLists.txt:
Avoid pointless recompilation of source files
in client library if possible. Merge static
libs (dbug, mysys) to create static client
library.
libmysqld/CMakeLists.txt:
Avoid pointless recompilation of source files
when building embedded. Instead, merge dbug and
mysys (and some other static libs) into embedded.
libmysqld/examples/CMakeLists.txt:
Embedded compilation on Unix
libmysqld/lib_sql.cc:
Do not define THD::clear_error() in lib_sql.cc
for embedded. Instead, use the same inline
definition from sql_class.h as in none-embedded
case (fixes duplicate symbol errors on Windows
and removes pointless #ifdef EMBEDDED)
man/CMakeLists.txt:
Install man files.
man/Makefile.am:
Add new file.
mysql-test/CMakeLists.txt:
Install mysql-test files
mysql-test/Makefile.am:
Add new files
mysql-test/lib/My/ConfigFactory.pm:
Allow testing with mtr in out-of-source builds.
mysql-test/lib/My/Find.pm:
the build configurations are now also available on Unix
Xcode on Mac uses the Release, RelwithDebinfo and Debug
subdirectories for executables. Earlier, build configurations
were available only on Windows.
mysql-test/lib/My/SafeProcess.pm:
Allow testing with mtr in out-of-source builds.
mysql-test/lib/My/SafeProcess/CMakeLists.txt:
Port CMakeLists.txt to Unix
mysql-test/lib/My/SafeProcess/safe_kill_win.cc:
add stdlib.h (to be able to compile with -DWIN32_LEAN_AND_MEAN)
mysql-test/lib/My/SafeProcess/safe_process_win.cc:
Add stdlib.h (to be able to compile with -DWIN32_LEAN_AND_MEAN)
define JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE if not defined
(can happen using MinGW compiler that comes with old SDK)
mysql-test/mtr.out-of-source:
Allow testing with mtr in out-of-source builds.
mysql-test/mysql-test-run.pl:
Allow testing with mtr in out-of-source builds.
Use common find_plugin macro for Windows and unix.
mysql-test/t/fulltext_plugin.test:
This test can now run on Windows as well.
mysys/CMakeLists.txt:
Port CMakeLists.txt to Unix
mysys/my_create.c:
config-win.h is dead => NO_OPEN_3 is never defined.
mysys/my_getsystime.c:
config-win.h is dead => define OFFSET_TO_EPOCH where it is used.
mysys/my_winthread.c:
Add win32 pthread_cancel - used by daemon_example
mysys/mysys_priv.h:
config-win.h is dead => include <sys/stat.h> where it is used
fix prototype of my_win_(f)stat
plugin/daemon_example/CMakeLists.txt:
Compile daemon_example with CMake
plugin/daemon_example/Makefile.am:
Add new file
plugin/fulltext/CMakeLists.txt:
Compile full-text example with CMake
plugin/fulltext/Makefile.am:
Add new file.
plugin/semisync/CMakeLists.txt:
Fix semisync to use common macro for plugins.
regex/CMakeLists.txt:
Use absolute filenames, when adding regex library
(we recompile files in embedded, and want to locate
sources via GET_TARGET_PROPERTY( ... SOURCES ..))
regex/regex2.h:
Remove pointless typedef (produces error with MinGW compiler)
scripts/CMakeLists.txt:
Add configure/install for scripts
sql-bench/CMakeLists.txt:
install sql-bench files
sql-bench/Makefile.am:
Add new file
sql/CMakeLists.txt:
Port CmakeLists.txt to Unix
sql/nt_servc.cc:
compile server with -DWIN32_LEAN_AND_MEAN
sql/share/CMakeLists.txt:
Install charsets
sql/share/Makefile.am:
Add new file
sql/sql_builtin.cc.in:
Handle pure-C plugins on Windows.
sql/sql_class.h:
Use the same clear_error macro in embedded and not embedded.
Fixes pointless #ifdef and avoids duplicate symbols when linking
on Windows.
storage/Makefile.am:
storage/mysql_storage_engine.cmake => cmake/plugin.cmake
storage/archive/CMakeLists.txt:
Add names for static and dynamic plugin libraries.
Link archive with zlib
storage/blackhole/CMakeLists.txt:
Add names for static and dynamic storage
engine libraries
storage/csv/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/example/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/federated/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/heap/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/ibmdb2i/CMakeLists.txt:
Better port for ibmdb2i plugin
storage/innobase/CMakeLists.txt:
Run system checks.
Add names for static and dynamic storage engine
libraries.
storage/innobase/include/page0page.ic:
Fix compile error on OpenSolaris.
storage/myisam/CMakeLists.txt:
Port CmakeLists.txt to Unix
storage/myisammrg/CMakeLists.txt:
Add names for static and dynamic storage engine
libraries
storage/mysql_storage_engine.cmake:
storage/mysql_storage_engine.cmake => cmake/plugin.cmake
support-files/CMakeLists.txt:
Configure and install some files from support-files.
support-files/Makefile.am:
Add new file
tests/CMakeLists.txt:
In general case, mysqlclient library can be dependent
on C++ runtime(if it includes yassl and is not compiled
with gcc or MSVC)
unittest/mysys/CMakeLists.txt:
Add unit tests
unittest/mysys/Makefile.am:
Add new file
unittest/mytap/CMakeLists.txt:
Add library for unit tests
unittest/mytap/Makefile.am:
Add new file
unittest/mytap/tap.c:
fix function definitions to match declarations
win/create_def_file.js:
Fix link error with intel compiler (icl
defines of special label for exception handler)
This fixes the regression introduced in 5.1 that prevents 64 bit builds on Intel while still keeping the core2 hack operational so the cluster can build.
bzr branch mysql-5.1-performance-version mysql-trunk # Summit
cd mysql-trunk
bzr merge mysql-5.1-innodb_plugin # which is 5.1 + Innodb plugin
bzr rm innobase # remove the builtin
Next step: build, test fixes.
platforms and emphasize the fact that gcc is the default
compiler on all platforms.
Make it possible to say --compiler=SunStudio instead of
--compiler=forte
Fix a bug that lost an important flag for C++ builds.
for those cases when gcc, cc-5.0, icpc or icc isn't in the path.
Fixed handling of 32 and 64 bits.
Downgraded Solaris builds on Forte to use -xO2 rather than -xO3.
Made it possible to build 64-bits on Mac OS X
Fixed some bugs in setting CC, CXX, ASFLAGS, LDFLAGS
Fixed bugs relating to use of SunStudio/Forte in check-cpu
Reorganized code a bit
Removed the use of --with-fast-mutexes since they aren't really
fast, rather slow.
Added -static-libgcc when using gcc
Added optimising compilation flags on BSD
Added use of curses library on Solaris
Removed the use of MY_ATOMIC_MODE_RWLOCKS which removed use of
atomic instructions
Added support for Forte on Solaris/x86
The convert_constant_item function converts a constant to integer using
field for condition like 'field = a_constant'. In some cases the
convert_constant_item is called for a subquery when outer select is already
being executed, so convert_constant_item saves field's value to prevent its
corruption. For EXPLAIN and at the prepare phase field's value isn't
initialized yet, thus when convert_constant_item tries to restore saved
value it fails assertion.
Now the convert_constant_item doesn't save/restore field's value if it's
haven't been read yet. Outer constant values are always saved.
mysql-test/r/explain.result:
Added a test case for the bug#37870.
mysql-test/t/explain.test:
Added a test case for the bug#37870.
sql/item_cmpfunc.cc:
Bug#37870: Usage of uninitialized value caused failed assertion.
Now the convert_constant_item doesn't save/restore field's value if it's
haven't been read yet. Outer constant values are always saved.
Link with mtmalloc on Solaris as it is done in our release builds.
Replace deprecated flag with the newer option as already done in
other scripts.
BUILD/compile-solaris-amd64-forte:
Link to mtmalloc and replace deprecated flag.
BUILD/compile-solaris-amd64-forte-debug:
Replace deprecated flag.
BUILD/compile-solaris-sparc:
Link to mtmalloc.
BUILD/compile-solaris-sparc-forte:
Link to mtmalloc
Post-merge fix: Add those files which are new in version 5.1
This is the 5.1 version of the fix for
Bug #34291 compile-amd64-debug-max-no-ndb missing
BUILD/Makefile.am:
Post-merge fix: Add those files which are new in version 5.1
